Dental and medical students’ self-directed learning and motivation
An evaluation of two multiple-choice questions systems using machine learning
DOI:
https://doi.org/10.7146/lom.v17i29.140337Nøgleord:
MCQ, Machine Learning, Dental, Motivation, Self-Regulated LearningResumé
This comparative case study reports a study investigating student evaluation of Multiple-Choice questions (MCQ) through machine learning as a means of learning. The focus is on self-directed learning and motivation. The study evaluates two systems developed at Aarhus University: "MED MCQ" used by medical students, and "MCQ anatomy" used by dental students. The study evaluates two surveys in SurveyXact with responses from 126 medical students and 70 dental students. We use topic modeling over free text responses. The machine learning model identifies two groups of students who, in different ways, experience interacting with the system as motivating and facilitating their learning process. The students' experience increases self-directed learning by being able to choose the form of presentation of questions and answer questions independently of the instructor. The article discusses how educators and developers can use MCQs to promote student learning and how to analyze open-ended questions with machine learning.
Downloads
Referencer
Ang, E. T., Chan, J. M., Gopal, V., & Li Shia, N. (2018). Gamifying anatomy education. Clinical Anatomy, 31(7), 997-1005.
Armstrong, R. D., Jones, D. H., Koppel, N. B., & Pashley, P. J. (2004). Computerized adaptive testing with multiple-form structures. Applied Psychological Measurement, 28(3), 147-164.
Bergstrom, B. A., & Lunz, M. E. (1999). CAT for certification and licensure. Innovations in computerized assessment, 67-91.
Billings, M., DeRuchie, K., Hussie, K., Kulesher, A., Merrell, J., Morales, A., Paniagua, M.,
Sherlock, J., Swygert, K., & Tyson, J. (2021). NBME item writing guide. Constructing written test questions for the health sciences. Philadelphia: National Board of Medical Examiners.
Bjork, R. A., & Bjork, E. L. (2019). Forgetting as the friend of learning: Implications for teaching and self-regulated learning. Advances in Physiology Education, 43(2), 164-167.
Borakati, A. (2021). Evaluation of an international medical E-learning course with natural language processing and machine learning. BMC Med Educ, 21, 1-10.Chandra, S., Katyal, R., Chandra, S., Singh, K., Singh, A., & Joshi, H. (2018). Medical education/original article creating valid multiple-choice questions (MCQs) Bank with Faculty Development of Pharmacology. Indian J Physiol Pharmacol, 62(3), 359-366.
Deci, E. L., & Ryan, R. M. (2012). Self-determination theory. Handbook of theories of social psychology, 1(20), 416-436.
Decorte, T., Malm, A., Sznitman, S. R., Hakkarainen, P., Barratt, M. J., Potter, G. R., Werse, B., Kamphausen, G., Lenton, S., & Asmussen Frank, V. (2019). The challenges and benefits of analyzing feedback comments in surveys: Lessons from a cross-national online survey of small-scale cannabis growers. Methodological innovations, 12(1), 205979911982560. https://doi.org/10.1177/2059799119825606
dos Santos, J. A., Syed, T. I., Naldi, M. C., Campello, R. J., & Sander, J. (2019). Hierarchical density-based clustering using MapReduce. IEEE Transactions on Big Data, 7(1), 102-114
Downing, S. M., & Haladyna, T. M. (2004). Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ, 38(3), 327-333.
Drasgow, F., & Olson-Buchanan, J. B. (1999). Innovations in computerized assessment. L. Erlbaum Associates, Publishers.
Durosaro, O., Lachman, N., & Pawlina, W. (2008). Use of knowledge-sharing web-based portal in gross and microscopic anatomy. Annals Academy of Medicine Singapore, 37(12), 998.
Edmondson, D. R., Boyer, S. L., & Artis, A. B. (2012). Self-directed learning: A meta-analytic review of adult learning constructs. International Journal of Education Research, 7(1), 40-48.
Einig, S. (2013). Supporting Students' Learning: The Use of Formative Online Assessments. Accounting education (London, England), 22(5), 425-444. https://doi.org/10.1080/09639284.2013.803868
Eleanor, S., & Mick, P. C. (2017). Some Methodological Uses of Responses to Open Questions and Other Verbatim Comments in Quantitative Surveys. Methoden, daten, analysen, 11(2), 115-135. https://doi.org/10.12758/mda.2017.01
Evergreen, S., & Metzner, C. (2013). Design Principles for Data Visualization in Evaluation: Design Principles for Data Visualization. New directions for evaluation, 2013(140), 5-20. https://doi.org/10.1002/ev.20071
Ghojogh, B., Crowley, M., Karray, F., & Ghodsi, A. (2023). Uniform Manifold Approximation and Projection (UMAP). I B. Ghojogh, M. Crowley, F. Karray & A. Ghodsi (red.), Elements of Dimensionality Reduction and Manifold Learning (s. 479-497). Springer International Publishing. https://doi.org/10.1007/978-3-031-10602-6_17
Grolnick, W. S., & Ryan, R. M. (1987). Autonomy in children's learning: an experimental and individual difference investigation. Journal of personality and social psychology, 52(5), 890.
Guay, F., Ratelle, C. F., & Chanal, J. (2008). Optimal Learning in Optimal Contexts: The Role of Self-Determination in Education. Canadian psychology = Psychologie canadienne, 49(3), 233-240. https://doi.org/10.1037/a0012758
Hendricson, W. D., Andrieu, S. C., Chadwick, D. G., Chmar, J. E., Cole, J. R., George, M. C., Glickman, G. N., Glover, J. F., Goldberg, J. S., Change, A. C. o., & Education, I. i. D. (2006). Educational strategies associated with development of problem‐solving, critical thinking, and self‐directed learning. J Dent Educ, 70(9), 925-936.
Javaeed, A. (2018). Assessment of higher ordered thinking in medical education: multiple choice questions and modified essay questions. MedEdPublish, 7, 128.
Kanzow, P., Schmidt, D., Herrmann, M., Wassmann, T., Wiegand, A., & Raupach, T. (2023). Use of Multiple-Select Multiple-Choice Items in a Dental Undergraduate Curriculum: Retrospective Study Involving the Application of Different Scoring Methods. JMIR Med Educ, 9, e43792. https://doi.org/10.2196/43792
Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, 18-33. https://doi.org/https://doi.org/10.1016/j.compedu.2016.10.001
Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers.
Lee, N. W., Shamsuddin, W. N. F. W., Wei, L. C., Anuardi, M. N. A. M., Heng, C. S., & Abdullah, A. N. (2021). Using Online Multiple Choice Questions with Multiple Attempts: A Case for Self-Directed Learning among Tertiary Students. International Journal of Evaluation and Research in Education, 10(2), 553-568.
Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological science, 23(11), 1337-1344.
Malzer, C., & Baum, M. (2020). A Hybrid Approach To Hierarchical Density-based Cluster Selection. Ithaca.
Mazmanian, P., & Feldman, M. (2011). Theory is needed to improve education, assessment and policy in self‐directed learning. Med Educ, 45(4), 324-326.
McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European journal of cognitive psychology, 19(4-5), 494-513.
McDermott, K. B., Agarwal, P. K., D'Antonio, L., Roediger III, H. L., & McDaniel, M. A. (2014). Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of Experimental Psychology: Applied, 20(1), 3.
McInnes, L., Healy, J., & Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.
Merriam, S. B. (2001). Andragogy and self-directed learning: Pillars of adult learning theory. New directions for adult and continuing education, 2001(89), 3.
Murray, J. H. (1997). Hamlet on the holodeck : the future of narrative in cyberspace. MIT Press.
Nicol, D. (2007). E-assessment by design: using multiple-choice tests to good effect. Journal of further and higher education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922
Pai, K. M., Rao, K. R., Punja, D., & Kamath, A. (2014). The effectiveness of self-directed learning (SDL) for teaching physiology to first-year medical students. The Australasian medical journal, 7(11), 448.
Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ, 7(1), 1-7.
Ryan, R. M., & Deci, E. L. (2000). Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being. The American psychologist, 55(1), 68-78. https://doi.org/10.1037/0003-066X.55.1.68 (Positive Psychology)
Say, R., Visentin, D., Cummings, E., Carr, A., & King, C. (2022). Formative online multiple-choice tests in nurse education: An integrative review. Nurse education in practice, 58, 103262.
Schuwirth, L. W. T., & Van Der Vleuten, C. P. M. (2004). Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ, 38(9), 974-979. https://doi.org/10.1111/j.1365-2929.2004.01916.x
Stake, R. E. (2013). Multiple case study analysis. Guilford press.
Tam, M., Hart, A., Williams, S., Holland, R., Heylings, D., & Leinster, S. (2010). Evaluation of a computer program (‘disect’) to consolidate anatomy knowledge: A randomised-controlled trial. Med Teach, 32(3), e138-e142.
Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K. M., & Deci, E. L. (2004). Motivating learning, performance, and persistence: the synergistic effects of intrinsic goal contents and autonomy-supportive contexts. Journal of personality and social psychology, 87(2), 246.
Yin, R. K. (2009). Case study research: Design and methods (årg. 5). sage.
Yu, D., & Xiang, B. (2023). Discovering topics and trends in the field of Artificial Intelligence: Using LDA topic modeling. Expert systems with applications, 225, 120114. https://doi.org/10.1016/j.eswa.2023.120114
Zia, S., Jabeen, F., Atta, K., & Sial, N. A. (2016). Self-directed learning (SDL), an effective method for teaching physiology to medical students. PJMHS, 10(3), 699-702.
Zimmerman, B. J. (2002). Becoming a Self-Regulated Learner: An Overview. Theory into practice, 41(2), 64-70. https://doi.org/10.1207/s15430421tip4102_2
Zimmerman, B. J. (2008). Investigating Self-Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects. American educational research journal, 45(1), 166-183. https://doi.org/10.3102/0002831207312909
Zimmerman, B. J., & Schunk, D. H. (2011). Self-regulated learning and performance: An introduction and an overview. I Handbook of self-regulation of learning and performance. (s. 1-12). Routledge/Taylor & Francis Group.
Downloads
Publiceret
Citation/Eksport
Nummer
Sektion
Licens
Copyright (c) 2023 Emilie Leth Rasmussen, Malthe Have Musaeus, Mads Ronald Dahl, Henrik Løvschall, Peter Musaeus
Dette værk er under følgende licens Creative Commons Navngivelse – Ikke-kommerciel – Ingen Bearbejdede Værker (by-nc-nd).
Artikler publiceret i Tidsskriftet for Læring og Medier er licenseret under en Creative Commons Navngivelse-IkkeKommerciel-IngenBearbejdelse 4.0 Unported Licens.
Forfattere bevarer deres ophavsret og giver tidsskriftet ret til første publicering, samtidigt med at værket er omfattet af Creative Commons Attribution-licensen: Navngivelse – Ikke-kommerciel - Ingen Bearbejdede Værker (by-nc-nd). Læs om licensen på http://www.creativecommons.dk/om/.
---
På LOM.dk kan du endvidere finde artikler fra det nu nedlagte Tidsskrift for Universiteternes Efter- og Videreuddannelse (UNEV). Vær opmærksom på, at der gælder særlige regler for UNEV artikler:
Det er forfatterne og evt. andre ophavsret indehavere, der har ophavsretten til artikler udgivet i UNEV regi, og det er en betingelse for adgang til artiklerne, at brugere anerkender og overholder de juridiske retningslinjer forbundet hermed.
- Brugere må downloade og printe én kopi af en hvilken som helst UNEV artikel mhp. private studier eller forskning.
- Det er ikke tilladt at videredistribuere artikler eller anvende disse til indtægtsdækkede aktiviteter eller kommercielle formål.
- Det er tilladt at distribuere URL’en til UNEV artikler.
