Dental and medical students’ self-directed learning and motivation

An evaluation of two multiple-choice questions systems using machine learning


  • Emilie Leth Rasmussen Aarhus Universitet
  • Malthe Have Musaeus It-Universitet København
  • Mads Ronald Dahl Institution Centre for Educational Development, Aarhus Universitet
  • Henrik Løvschall Deparment for Dentistry and Oral Health, Aarhus Universitet
  • Peter Musaeus Institution Centre for Educational Development, Aarhus Universitet



MCQ, Machine Learning, Dental, Motivation, Self-Regulated Learning


This comparative case study reports a study investigating student evaluation of Multiple-Choice questions (MCQ) through machine learning as a means of learning. The focus is on self-directed learning and motivation. The study evaluates two systems developed at Aarhus University: "MED MCQ" used by medical students, and "MCQ anatomy" used by dental students. The study evaluates two surveys in SurveyXact with responses from 126 medical students and 70 dental students. We use topic modeling over free text responses. The machine learning model identifies two groups of students who, in different ways, experience interacting with the system as motivating and facilitating their learning process. The students' experience increases self-directed learning by being able to choose the form of presentation of questions and answer questions independently of the instructor. The article discusses how educators and developers can use MCQs to promote student learning and how to analyze open-ended questions with machine learning.


Download-data er endnu ikke tilgængelig.


Malthe Have Musaeus, It-Universitet København

Bachelor student in Data Science

It-Universitet København

Mads Ronald Dahl, Institution Centre for Educational Development, Aarhus Universitet

MSc biotechnology, Ph.D, Master in Informatics, Special konsultent.

Centre for Educational Development, Aarhus Universitet

Henrik Løvschall, Deparment for Dentistry and Oral Health, Aarhus Universitet

Cand odont, Ph. D, Lektor.

Deparment for Dentistry and Oral Health, Aarhus Universitet

Peter Musaeus, Institution Centre for Educational Development, Aarhus Universitet

Cand psych, Ph. D, Lektor

Institution Centre for Educational Development, Aarhus Universitet


Ang, E. T., Chan, J. M., Gopal, V., & Li Shia, N. (2018). Gamifying anatomy education. Clinical Anatomy, 31(7), 997-1005.

Armstrong, R. D., Jones, D. H., Koppel, N. B., & Pashley, P. J. (2004). Computerized adaptive testing with multiple-form structures. Applied Psychological Measurement, 28(3), 147-164.

Bergstrom, B. A., & Lunz, M. E. (1999). CAT for certification and licensure. Innovations in computerized assessment, 67-91.

Billings, M., DeRuchie, K., Hussie, K., Kulesher, A., Merrell, J., Morales, A., Paniagua, M.,

Sherlock, J., Swygert, K., & Tyson, J. (2021). NBME item writing guide. Constructing written test questions for the health sciences. Philadelphia: National Board of Medical Examiners.

Bjork, R. A., & Bjork, E. L. (2019). Forgetting as the friend of learning: Implications for teaching and self-regulated learning. Advances in Physiology Education, 43(2), 164-167.

Borakati, A. (2021). Evaluation of an international medical E-learning course with natural language processing and machine learning. BMC Med Educ, 21, 1-10.Chandra, S., Katyal, R., Chandra, S., Singh, K., Singh, A., & Joshi, H. (2018). Medical education/original article creating valid multiple-choice questions (MCQs) Bank with Faculty Development of Pharmacology. Indian J Physiol Pharmacol, 62(3), 359-366.

Deci, E. L., & Ryan, R. M. (2012). Self-determination theory. Handbook of theories of social psychology, 1(20), 416-436.

Decorte, T., Malm, A., Sznitman, S. R., Hakkarainen, P., Barratt, M. J., Potter, G. R., Werse, B., Kamphausen, G., Lenton, S., & Asmussen Frank, V. (2019). The challenges and benefits of analyzing feedback comments in surveys: Lessons from a cross-national online survey of small-scale cannabis growers. Methodological innovations, 12(1), 205979911982560.

dos Santos, J. A., Syed, T. I., Naldi, M. C., Campello, R. J., & Sander, J. (2019). Hierarchical density-based clustering using MapReduce. IEEE Transactions on Big Data, 7(1), 102-114

Downing, S. M., & Haladyna, T. M. (2004). Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ, 38(3), 327-333.

Drasgow, F., & Olson-Buchanan, J. B. (1999). Innovations in computerized assessment. L. Erlbaum Associates, Publishers.

Durosaro, O., Lachman, N., & Pawlina, W. (2008). Use of knowledge-sharing web-based portal in gross and microscopic anatomy. Annals Academy of Medicine Singapore, 37(12), 998.

Edmondson, D. R., Boyer, S. L., & Artis, A. B. (2012). Self-directed learning: A meta-analytic review of adult learning constructs. International Journal of Education Research, 7(1), 40-48.

Einig, S. (2013). Supporting Students' Learning: The Use of Formative Online Assessments. Accounting education (London, England), 22(5), 425-444.

Eleanor, S., & Mick, P. C. (2017). Some Methodological Uses of Responses to Open Questions and Other Verbatim Comments in Quantitative Surveys. Methoden, daten, analysen, 11(2), 115-135.

Evergreen, S., & Metzner, C. (2013). Design Principles for Data Visualization in Evaluation: Design Principles for Data Visualization. New directions for evaluation, 2013(140), 5-20.

Ghojogh, B., Crowley, M., Karray, F., & Ghodsi, A. (2023). Uniform Manifold Approximation and Projection (UMAP). I B. Ghojogh, M. Crowley, F. Karray & A. Ghodsi (red.), Elements of Dimensionality Reduction and Manifold Learning (s. 479-497). Springer International Publishing.

Grolnick, W. S., & Ryan, R. M. (1987). Autonomy in children's learning: an experimental and individual difference investigation. Journal of personality and social psychology, 52(5), 890.

Guay, F., Ratelle, C. F., & Chanal, J. (2008). Optimal Learning in Optimal Contexts: The Role of Self-Determination in Education. Canadian psychology = Psychologie canadienne, 49(3), 233-240.

Hendricson, W. D., Andrieu, S. C., Chadwick, D. G., Chmar, J. E., Cole, J. R., George, M. C., Glickman, G. N., Glover, J. F., Goldberg, J. S., Change, A. C. o., & Education, I. i. D. (2006). Educational strategies associated with development of problem‐solving, critical thinking, and self‐directed learning. J Dent Educ, 70(9), 925-936.

Javaeed, A. (2018). Assessment of higher ordered thinking in medical education: multiple choice questions and modified essay questions. MedEdPublish, 7, 128.

Kanzow, P., Schmidt, D., Herrmann, M., Wassmann, T., Wiegand, A., & Raupach, T. (2023). Use of Multiple-Select Multiple-Choice Items in a Dental Undergraduate Curriculum: Retrospective Study Involving the Application of Different Scoring Methods. JMIR Med Educ, 9, e43792.

Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, 18-33.

Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers.

Lee, N. W., Shamsuddin, W. N. F. W., Wei, L. C., Anuardi, M. N. A. M., Heng, C. S., & Abdullah, A. N. (2021). Using Online Multiple Choice Questions with Multiple Attempts: A Case for Self-Directed Learning among Tertiary Students. International Journal of Evaluation and Research in Education, 10(2), 553-568.

Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological science, 23(11), 1337-1344.

Malzer, C., & Baum, M. (2020). A Hybrid Approach To Hierarchical Density-based Cluster Selection. Ithaca.

Mazmanian, P., & Feldman, M. (2011). Theory is needed to improve education, assessment and policy in self‐directed learning. Med Educ, 45(4), 324-326.

McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European journal of cognitive psychology, 19(4-5), 494-513.

McDermott, K. B., Agarwal, P. K., D'Antonio, L., Roediger III, H. L., & McDaniel, M. A. (2014). Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of Experimental Psychology: Applied, 20(1), 3.

McInnes, L., Healy, J., & Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.

Merriam, S. B. (2001). Andragogy and self-directed learning: Pillars of adult learning theory. New directions for adult and continuing education, 2001(89), 3.

Murray, J. H. (1997). Hamlet on the holodeck : the future of narrative in cyberspace. MIT Press.

Nicol, D. (2007). E-assessment by design: using multiple-choice tests to good effect. Journal of further and higher education, 31(1), 53-64.

Pai, K. M., Rao, K. R., Punja, D., & Kamath, A. (2014). The effectiveness of self-directed learning (SDL) for teaching physiology to first-year medical students. The Australasian medical journal, 7(11), 448.

Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ, 7(1), 1-7.

Ryan, R. M., & Deci, E. L. (2000). Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being. The American psychologist, 55(1), 68-78. (Positive Psychology)

Say, R., Visentin, D., Cummings, E., Carr, A., & King, C. (2022). Formative online multiple-choice tests in nurse education: An integrative review. Nurse education in practice, 58, 103262.

Schuwirth, L. W. T., & Van Der Vleuten, C. P. M. (2004). Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ, 38(9), 974-979.

Stake, R. E. (2013). Multiple case study analysis. Guilford press.

Tam, M., Hart, A., Williams, S., Holland, R., Heylings, D., & Leinster, S. (2010). Evaluation of a computer program (‘disect’) to consolidate anatomy knowledge: A randomised-controlled trial. Med Teach, 32(3), e138-e142.

Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K. M., & Deci, E. L. (2004). Motivating learning, performance, and persistence: the synergistic effects of intrinsic goal contents and autonomy-supportive contexts. Journal of personality and social psychology, 87(2), 246.

Yin, R. K. (2009). Case study research: Design and methods (årg. 5). sage.

Yu, D., & Xiang, B. (2023). Discovering topics and trends in the field of Artificial Intelligence: Using LDA topic modeling. Expert systems with applications, 225, 120114.

Zia, S., Jabeen, F., Atta, K., & Sial, N. A. (2016). Self-directed learning (SDL), an effective method for teaching physiology to medical students. PJMHS, 10(3), 699-702.

Zimmerman, B. J. (2002). Becoming a Self-Regulated Learner: An Overview. Theory into practice, 41(2), 64-70.

Zimmerman, B. J. (2008). Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects. American educational research journal, 45(1), 166-183.

Zimmerman, B. J., & Schunk, D. H. (2011). Self-regulated learning and performance: An introduction and an overview. I Handbook of self-regulation of learning and performance. (s. 1-12). Routledge/Taylor & Francis Group.





Leth Rasmussen, E., Have Musaeus, M., Dahl, M. R., Løvschall, H., & Musaeus, P. (2024). Dental and medical students’ self-directed learning and motivation: An evaluation of two multiple-choice questions systems using machine learning. Tidsskriftet Læring Og Medier (LOM), 17(29).



LOM29: Motivation, agens og teknologi