Does the format matter? How the multiple-choice format might complicate the MKT items
DOI:
https://doi.org/10.7146/nomad.v16i4.148408Abstract
In order to design appropriate professional development programs for teachers, an instrument has been developed in the U.S. to measure teachers’ mathematical knowledge for teaching. The process of translating and adapting these measures for use in other countries involves several challenges. This article focuses on issues related to the multiple-choice format of the items. Analyses of focus-group interviews reveal that the multiple-choice format may complicate the items. The teachers’ reflections about the format in this Norwegian case contribute to the understanding of this important challenge.
References
Ball, D. L. & Hill, H. C. (2008). Mathematical knowledge for teaching (MKT) measures. Mathematics released items 2008. Retrieved September 3, 2011, from http://sitemaker.umich.edu/lmt/files/LMT_sample_items.pdf
Ball, D. L., Thames, M. H. & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59 (5), 389-407. https://doi.org/10.1177/0022487108324554
Bergh, H. van den (1990). On the construct validity of multiple-choice items for reading comprehension. Applied Psychological Measurement, 14 (1), 1-12. https://doi.org/10.1177/014662169001400101
Boaler, J. (2000). Introduction: intricacies of knowledge, practice, and theory. In J. Boaler (Ed.), Multiple perspectives on mathematics teaching & learning (pp. 1-17). Westport: Greenwood Publishing Group. https://doi.org/10.5040/9798400688362.0004
Boodoo, G. M. (1993). Performance assessments or multiple choice? Educational Horizons, 72 ( 1), 55-56.
Bryman, A. (2004). Social research methods (2nd ed.). New York: Oxford University Press Inc.
Burton, S. J., Sudweeks, R. R., Merrill, P. F. & Wood, B. (1991). How to prepare better multiple-choice test items: guidelines for university faculty. Brigham Young University Testing Services & The Department of Instructional Science. Retrieved February 13, 2012, from https://www2.kumc.edu/comptraining/ documents/WrittingbetterMCtestitems1991BrighamYoung.pdf
Clauser, B. E. & Margolis, M. J. (2006). Testing for licensure and certification in the professions. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 701-731). Santa Barbara: Greenwood Publishing Group.
Cole, Y. A. (2011). Mathematical knowledge for teaching: exploring its transferability and measurement in Ghana (Unpublished doctoral dissertation). Ann Arbor: University of Michigan.
Delaney, S. F. (2008). Adapting and using U.S. measures to study Irish teachers' mathematical knowledge for teaching (Unpublished doctoral dissertation). Ann Arbor: University of Michigan.
Delaney, S., Ball, D., Hill, H., Schilling, S. & Zopf, D. (2008). "Mathematical knowledge for teaching": adapting U.S. measures for use in Ireland. Journal of Mathematics Teacher Education, 11 (3), 171-197. https://doi.org/10.1007/s10857-008-9072-1
Fauskanger, J. & Mosvold, R. (2010). Undervisningskunnskap i matematikk: Tilpasning av en amerikansk undersøkelse til norsk, og læreres opplevelse av undersøkelsen. Norsk Pedagogisk Tidsskrift, 94 (2), 112-123. https://doi.org/10.18261/ISSN1504-2987-2010-02-03
Fauskanger, J., Jakobsen, A., Mosvold, R. & Bjuland, R. (in press). Analysis of psychometric properties as part of an ongoing adaptation process of MKT items for use in other countries. ZDM The International Journal on Mathematics Education, 44 (2). https://doi.org/10.1007/s11858-012-0403-4
Fraenkel, J. R. & Wallen, N. E. (2006). How to design and evaluate research in education (6th ed.). New York: McGraw-Hill.
Goodchild, S. (2001). Students' goals: a case study of activity in a mathematics classroom. Bergen: Caspar Forlag.
Graeber, A. & Tirosh, D. (2008). Pedagogical content knowledge. Useful concept or elusive notion. In P. Sullivan & T. Wood (Eds.), The international handbook of mathematics teacher education (Vol. 1., pp. 117-132). Rotterdam/ Taipei: Sense Publishers.
Haertel, E. (2004). Interpretive argument and validity argument for certification testing: Can we escape the need for psychological theory? Measurement: Interdisciplinary Research and Perspectives, 2 (3), 175-178.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). New Jersey: Lawrence Earlbaum Associates. https://doi.org/10.4324/9780203825945
Hambleton, R. K. & Patsula, L. (1998). Adapting tests for use in multiple languages and cultures. Social Indicators Research, 45 (1-3), 153-171. https://doi.org/10.1023/A:1006941729637
Hill, H. C. (2010). The nature and predictors of elementary teachers' mathematical knowledge for teaching. Journal for Research in Mathematics Education, 41 (5), 513-545. https://doi.org/10.5951/jresematheduc.41.5.0513
Hill, H. C., Ball, D. L., Blunk, M., Goffney, I. M. & Rowan, B. (2007). Validating the ecological assumption: the relationship of measure scores to classroom teaching and student learning. Measurement: Interdisciplinary Research and Perspectives, 5 (2&3), 107-118. https://doi.org/10.1080/15366360701487138
Hill, H. C., Ball, D. L. & Schilling, S. (2008). Unpacking "pedagogical content knowledge": conceptualizing and measuring teachers' topic-specific knowledge of students. Journal for Research in Mathematics Education, 39 (4), 372-400. https://doi.org/10.5951/jresematheduc.39.4.0372
Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L. et al. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: an exploratory study. Cognition and Instruction, 26 (4), 430-511. https://doi.org/10.1080/07370000802177235
Hill, H. C., Dean, C. & Goffney, I. M. (2007). Assessing elemental and structural validity: data from teachers, non-teachers, and mathematicians. Measurement: Interdisciplinary Research and Perspectives, 5 (2), 81-92. https://doi.org/10.1080/15366360701486999
Hill, H. C., Rowan, B. & Ball, D. L. (2005). Effects of teachers' mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42 (2), 371-406. https://doi.org/10.3102/00028312042002371
Hill, H. C., Sleep, L., Lewis, J. M. & Ball, D. L. (2007). Assessing teachers' mathematical knowledge. What knowledge matters and what evidence counts? In F. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 111-156). Charlotte: Information Age Publishing.
Hollingworth, L., Beard, J. J. & Proctor, T. P. (2007). An investigation of item type in a standards-based assessment. Practical Assessment Research & Evaluation, 12 (18). Retrieved April 3, 2011, from http://pareonline.net/pdf/ v12n18.pdf
Jakobsen, A., Fauskanger, J., Mosvold, R. & Bjuland, R. (2011). Comparison of item performance in a Norwegian study using US developed mathematical knowledge for teaching measures. In M. Pytlak, T. Rowland & E. Swoboda (Eds.), Proceedings of the Seventh Congress of the European Society for Research in Mathematics Education (pp. 1802-1811). University of Rzeszów, Poland.
Kwon, M. (2009). Validating the adapted mathematical knowledge for teaching (MKT) measures in Korea. Paper presented at the AERA 2009 Annual Meeting.
Lysne, A. (2006). Assessment theory and practice of students' outcomes in Nordic countries. Scandinavian Journal of Educational Research, 50 (3), 327-359. https://doi.org/10.1080/00313830600743365
Ma, L. (2010). Knowing and teaching elementary mathematics: teachers' understanding of fundamental mathematics in China and the United States. Anniversary edition. New York: Routledge. https://doi.org/10.4324/9780203856345
Martinez, M. E. (1999). Cognition and the question of test item format. Educational Psychologist, 34 (4), 207-218. https://doi.org/10.1207/s15326985ep3404_2
Morris, A. K., Hiebert, J. & Spitzer, S. M. (2009). Mathematical knowledge for teaching in planning and evaluating instruction: What can preservice teachers learn? Journal for Research in Mathematics Education, 40 (5), 491-529. https://doi.org/10.5951/jresematheduc.40.5.0491
Mosvold, R., Fauskanger, J., Jakobsen, A. & Melhus, K. (2009). Translating test items into Norwegian - without getting lost in translation? Nordic Studies in Mathematics Education, 14 (4), 9-31.
Ng, D. (2012). Using the MKT measures to reveal Indonesian teachers' mathematical knowledge: challenges and potentials. ZDM The International Journal on Mathematics Education, 44 (2). (doi: 10.1007/s11858-011-0375-9) https://doi.org/10.1007/s11858-011-0375-9
Ng, D., Mosvold, R. & Fauskanger, J. (2012). Translating and adapting the mathematical knowledge for teaching (MKT) measures: the cases of Indonesia and Norway. The Mathematics Enthusiast, 9 (1&2), 149-178. https://doi.org/10.54870/1551-3440.1238
Schilling, S., Blunk, M. & Hill, H., C. (2007). Test validation and the MKT measures: generalizations and conclusions. Measurement: Interdisciplinary Research and Perspectives, 5 (2-3), 118-128. https://doi.org/10.1080/15366360701487146
Schoenfeld, A. H. (1992). Learning to think mathematically: problem solving, metacognition, and sense making in mathematics. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 334-370). New York: Macmillan.
Schoenfeld, A. H. (2007). Commentary: the complexities of assessing teacher knowledge. Measurement: Interdisciplinary research and perspectives, 5 (2), 198-204. https://doi.org/10.1080/15366360701492880
Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15 (2), 4-14. https://doi.org/10.2307/1175860
Singh, J. (1995). Measurement issues in cross-national research. Journal of International Business Studies, 26 (3), 597-619. https://doi.org/10.1057/palgrave.jibs.8490188
Sirnes, S. M. (2005). Flervalgsoppgaver - konstruksjon og analyse. Bergen: Fagbokforlaget.
Stigler, J. W. & Hiebert, J. (1999). The teaching gap: best ideas from the world's teachers for improving education in the classroom. New York: The Free Press.
Stylianides, A. J. & Delaney, S. (2011). The cultural dimension of teachers' mathematical knowledge. In T. Rowland & K. Ruthven (Eds.), Mathematical knowledge in teaching (pp. 179-191). London: Springer. https://doi.org/10.1007/978-90-481-9766-8_11
Sullivan, P. & Wood, B. (Eds.). (2008). Knowledge and beliefs in mathematics teaching and teaching development (Vol. 1). Rotterdam: Sense Publishers.
Tonheim, O. H. M. & Torkildsen, O. E. (2010). Matematikk 1 i lærerutdanninga - kvalifiserande? In P. Haug (Ed.), Kvalifisering til læreryrket (pp. 209-226). Oslo: Abstrakt forlag.
Törner, G., Rolka, K., Rösken, B. & Sriraman, B. (2010). Understanding a teacher's actions in the classroom by applying Schoenfeld's theory Teaching-In-Context: reflecting on goals and beliefs. In B. Sriraman & L. English (Eds.), Theories of mathematics education. Seeking new frontiers (pp. 401-420). Heidelberg: Springer. https://doi.org/10.1007/978-3-642-00742-2_38
Williams, J. (2011). Audit and evaluation of pedagogy: towards a cultural- historical perspective. In T. Rowland & K. Ruthven (Eds.), Mathematical knowledge in teaching (pp. 161-178). London: Springer. https://doi.org/10.1007/978-90-481-9766-8_10
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
