Analysis model for the subject specific use of text-image constructions in didactic learning materials
Analysis model that captures the subject-specific potential in writing-image constructs
DOI:
https://doi.org/10.7146/lt.v10i17.158795Keywords:
instructional materials, educational resources, multimodality, text-image-relations, disciplinary literacy, content area literacyAbstract
When a multimodal construction that combines image and text appears in a didactic learning material, it makes a difference when analyzing the image as a motivated sign. In learning material, such a sign will basically have a learning and teaching purpose. Therefore, it is essential to adopt a subject-disciplinary and subject-didactic focus in the analysis of the use of multimodal elements in didactic learning materials. This article presents a new method for analyzing how text-image compositions are - to a greater or lesser extent and for different didactic purposes - used to support learning and acquisition in didactic learning materials. Methodologically, theories and concepts from the multimodal theory of social semiotics, theory of disciplinary specificity in subject related texts and learning materials as well as concepts from learning material analysis are used. The article's contribution is methodological development and has relevance for student teachers, teachers and producers of teaching aids.
References
Adler, C. (1993). Directed picture processing: The effects for learners on recall of related text (Dissertation). University of South Florida.
Alkhateeb, M. (2019). Multiple Representations in 8th Grade Mathematics Textbook and the Extent to which Teachers Implement Them. International Electronic Journal of Mathematics Education, 14(1), 137-145. https://doi.org/10.12973/iejme/3982
Alvermann, D. E., & Wilson, A. A. (2011). Comprehension Strategy Instruction for Multimodal Texts in Science. Theory Into Practice, 50(2), 116–124. https://doi.org/10.1080/00405841.2011.558436
Bertin, J. (1983). Semiology of Graphics. The University of Wisconsin Press.
Bliss, T. (1990). Visuals in perspective: An analysis of U.S. history textbooks. Social Studies, 81, 10–10. doi:10.1080/00377996.1990.9957485
Boeriis, M. (2012). Tekstzoom: Om en dynamisk funktionel rangstruktur i visuelle tekster. In M. Boeriis & T. H. Andersen (Eds.), Nordisk Socialsemiotik: pædagogiske, multimodale og sprogvidenskabelige landvindinger (pp. 131-153). Syddansk Universitetsforlag.
Brugar, K. A., & Roberts, K. L. (2017). Seeing is believing: Promoting visual literacy in elementary social studies. Journal of Teacher Education, 68(3), 262–279. https://doi.org/10.1177/0022487117696280
Bundsgaard, J., & Hansen, T. I. (2011). Holistic evaluations of learning materials. In Rodríguez, J. R., Horsley, M., & S. V. Knudsen (Eds.), Local, National and Transnational Identities in Textbooks and Educational Media: Ten International Conference on Research on Textbooks and Educational Media September 2009 Santiago de Compostela – Spain (pp. 502-520). Santiago: IARTEM.
Butcher, K. R. (2006). Learning from text with diagrams: Promoting mental model development and inference generation. Journal of Educational Psychology, 98(1), 182–197. DOI: 10.1037/0022-0663.98.1.182.
Carney, R., & Levin, J. R. (2002). Pictorial Illustrations Still Improve Students’ Learning from Text. Educational Psychology Review, 14(1). 10.1023/A:1013176309260
Cheng, M.-C., Chou, P.-I., Wang, Y.-T., & Lin, C.-H. (2015). Learning Effects of a Science Textbook Designed with Adapted Cognitive Process Principles on Grade 5 Students. International Journal of Science and Mathematics Education, 13, 467–488. https://doi.org/10.1007/s10763-013-9492-8
Coleman, J. M., McTigue, E. M., & Smolkin, L. B. (2011). Elementary teachers’ use of graphical representations in science teaching. Journal of Science Teacher Education, 22, 613–643.
Danielsson, K. (2010). Learning Chemistry. Text Use and Text Talk in a Finland-Swedish Chemistry Classroom. IARTEM e-journal, 3(2), 1–28.
Danielsson, K., & Selander, S. (2016). Reading Multimodal Texts for Learning – a Model for Cultivating Multimodal Literacy. Designs for Learning, 8(1), 25–36. http://dx.doi.org/10.16993/dfl.72
Danielsson, K. & Selander, S. (2021). Multimodal Texts in Disciplinary Education: A Comprehensive Framework. Springer. https://doi.org/10.1007/978-3-030-63960-0
Devetak, I., Vogrinc, J., & Glažar, S. A. (2010). States of Matter Explanations in Slovenian Textbooks for Students Aged 6 to 14. The International Journal of Environmental and Science Education, 5(2), 217-235.
Fingeret, L. (2012). Visuals in children’s informational texts: A content analysis (Unpublished doctoral dissertation). Michigan State University. https://www.proquest.com/docview/1039317370?parentSessionId=URjPHmFSg4p17DTEpRuAaX6QbyQlf9KGIvcmZbCgGf8%3D&sourcetype=Dissertations%20&%20Theses
Fterniati, A. (2009). Teaching Multimodality in Greek Elementary School Language Arts. The International Journal of Learning, 16(4), 1-15.
Gissel, S. T. (2023). PISA 2022 Læsning: Delrapport [PISA 2022 Reading: interim report]. (1 udg.) VIVE Det Nationale Forsknings- og Analysecenter for Velfærd. https://www.uvm.dk/-/media/filer/uvm/int/231204-pisa-2022-laesning-pdf-ua.pdf
Gissel, S. T. (2026a). Typologi for læremidler og ressourcer, der kan bruges til læring eller undervisning [Typology for instructional resources and resources, that can be used for purposes of learning or teaching]. MONA - Matematik- Og Naturfagsdidaktik, 26(1). https://doi.org/10.7146/mona.v26i1.166713
Gissel, S. T. (2026b). Mediedidaktik i teori og praksis [Media didactics in theory and practice]. Copenhagen: Academica/Hans Reitzels forlag. 3rd edition.
Gissel, S. T., Buch, B., Oksbjerg, M., Kjeldsen, K., & Lytje, M. (2023). Lærervejledningslandskabet i Danmark [The teacher guide landscape in Denmark]. Learning Tech – Tidsskrift for læremidler, didaktik og teknologi, (12), 41-74. https://doi.org/10.7146/lt.v7i12.132029
Gissel, S., Carlsen, D., & Buch, B. (2025). Teachers’ Use of Learning Materials. In: Bagoly-Simó, P., Khaneboubi, M., Knight, B. (Eds.). Educational Media: International Perspectives. Oxford, United Kingdom: Peter Lang Verlag. Retrieved Dec 1, 2025, from https://doi.org/10.3726/b21556
Gissel, S. T., Carlsen, D., Kinnerup, L.B., Kjeldsen, K., Møller, M. & Oksbjerg, M. (2026a). Fagdidaktisk anvendelse af multimodalitet på danske digitale fagportaler. Signalement af samspillet mellem skrevet tekst og visuelle elementer i udvalgte fag. Learning Tech – Tidsskrift for læremidler, didaktik og teknologi, 17, 79-114.
Gissel, S. T., Kjeldsen, K., Carlsen, D., Oksbjerg, M., Møller, M., Nielsen, T., Kinnerup, L.B., & Johannesen, C. (2026b). A scoping review of research on disciplinary use of graphics-text-constructs in didactic learning materials. Learning Tech – Tidsskrift for læremidler, didaktik og teknologi, 17, 17-46.
Guo, D., Wright, K. L., & McTigue, E. M. (2018). A Content Analysis of Visuals in Elementary School Textbooks. The Elementary School Journal, 119(2), 245-265. https://doi.org/10.1086/700266
Halliday, M. A. K., & Martin, J. R. (1993). Writing science. Literacy and discursive power. University of Pittsburgh Press.
Hansen, J. J. (2006). Mellem design og didaktik: Om digitale læremidler i skolen. (Ph.d.-afhandling). Syddansk Universitet.
Hung, Y.-N. (2014). “What Are You Looking At?” An Eye Movement Exploration in Science Text Reading. International Journal of Science and Mathematics Education, 12, 241–260. DOI: 10.1007/s10763-013-9433-6
Iser, W. (1971). Indeterminacy and the Reader’s Response. In J. Hillis Miller (ed.), Aspects of Narrative: Selected Papers from the English Institute (pp. 1-45). Columbia University Press.
Jaeger, A. J., & Wiley, J. (2014). Do illustrations help or harm metacomprehension accuracy? Learning and Instruction, 34(6), 58–73. https://doi.org/10.1016/j.learninstruc.2014.08.002
Janko, T., & Knecht, P. (2014). Visuals in Geography Textbooks and Increasing the Reliability of a Research Instrument. In: P. Knecht, E.
Matthes, S. Schütze, & B. Aamotsbakken (Eds.), Methodologie und Methoden der Schulbuch- und Lehrmittelforschung (pp. 227–239). Klinkhardt.
Kapıcı, H. Ö., & Açıkalın, F. S. (2015). Examination of visuals about the particulate nature of matter in Turkish middle school science textbooks. Chemistry Education Research and Practice, 16, 518-536. https://doi.org/10.1039/C5RP00032G
Karlsson, A.-M. (2012). Multimodaliteten och språket: Om lexikogrammatikens styrkor och begränsningar i förhållande till några arbetslivsdiskurser. In M. Boeriis & T. H. Andersen (Eds.), Nordisk Socialsemiotik: pædagogiske, multimodale og sprogvidenskabelige landvindinger (pp. 17–38). Syddansk Universitetsforlag.
Knudsen, B., & Buttenschøn, M. (n.d.). Anden Verdenskrig [Second World War]. https://historie.alinea.dk/course/AAO3-anden-verdenskrig
Kress, G., & van Leeuwen, T. (2006). Reading Images. The Grammar of Visual Design. Routledge.
Lee, V. R. (2010). Adaptations and Continuities in the Use and Design of Visual Representations in US Middle School Science Textbooks. International Journal of Science Education, 32(8), 1099-1126.
Lenzner, A., Schnotz, W., & Müller, A. (2013). The role of decorative pictures in learning. Instructional Science, 41, 811-831. https://doi.org/10.1007/s11251-012-9256-z
Levin, J. R. (1981). On functions of pictures in prose. In F. J. Pirozzolo, and M. C. Wittrock (Eds.), Neuropsychological and Cognitive Processes in Reading (pp. 203–228). Academic Press.
Levin, J. R., Anglin, G. J., and Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D. M. Willows and H. A. Houghton (Eds.), The Psychology of Illustration: I. Basic Research (pp. 51–85). Springer.
Lim, F. V., & Toh, W. (2020). How to teach digital reading? Journal of Information Literacy, 14(2), pp. 24-43.
Liu, H. C., & Chuang, H. H. (2011). An examination of cognitive processing of multimedia information based on viewers’ eye movements. Interactive Learning Environments, 19(5), 503-517. https://doi.org/10.1080/10494820903520123
Mason, L., Pluchino, P. & Tornatora, M.C. (2015). Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning. Contemporary Educational Psychology, 41, pp.172–187. doi:10.1016/j.cedpsych.2015.01.004.
Mayer, R. E., Steinhoff, K., Bower, G., & Mars, R. (1995). A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text. Educational Technology Research and Development, 43(1), 31-43.
Mayer, R. E. (Ed). (2014a). Multimedia learning (2nd ed). Cambridge University Press.
Mayer, R. E. (2014b). Research Based Principles for Designing Multimedia Instruction. In Benassi, V. A., Overson, C., & Hakala, C. M. (Eds.), Applying science of learning in education: Infusing psychological science into the curriculum (pp. 59-70). http://teachpsych.org/ebooks/asle2014/index.php
Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82(4), 715–726. https://doi.org/10.1037/0022-0663.82.4.715
Mayer, R. E., Steinhoff, K., Bower, G., & Mars, R. (1995). A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text. Educational Technology Research and Development, 43(1), 31–43. https://doi.org/10.1007/BF02300480
McKean, B. (2002). Artistic Representations in Three Fifth-Grade Social Studies Textbooks. The Elementary School Journal, 103(2), 187-197. The University of Chicago Press. https://www.jstor.org/stable/1002234
Molina, A. I., Navarro, Ó., Ortega, M., & Lacruz, M. (2018). Evaluating multimedia learning materials in primary education using eye tracking. Computer Standards & Interfaces, 59, 45-60. https://doi.org/10.1016/j.csi.2018.02.004
Nikolajeva, M., Scott, C. (2000). The Dynamics of Picturebook Communication. Children’s Literature in Education, 31(4), 225–239. https://doi.org/10.1023/A:1026426902123
OECD. (2019). PISA 2018 Reading Framework. OECD Publishing.
OECD. (2023). PISA 2022 Results (Volume I): The State of Learning and Equity in Education. OECD Publishing. https://doi.org/10.1787/53f23881-en
Ohle-Peters, A., McElvany, N., & Ullrich, M. (2023). Students Reading Text-Picture-Material: The Role of Teacher Competence and Instructional Quality. International Journal of Educational Psychology, 12(3), 251-268. https://doi.org/10.17583/ijep.11558
Oruç, Ş., Uşurlu, N. B., & Tokcan, H. (2010). Using graphic illustrations with social studies textbooks. Procedia Social and Behavioral Sciences, 2, 1037–1042. https://doi.org/10.1016/j.sbspro.2010.03.146
Peirce, C. S. (1997). Collected Papers of Charles Sanders Peirce (C. Hartshorne & P. Weiss, Eds.). Thoemmes Press.
Peterson, M. (2016). Schemes for Integrating Text and Image in the Science Textbook: Effects on Comprehension and Situational Interest. International Journal of Environmental and Science Education, 11(6), 1365–1385.
Postigo, Y., & López-Manjón, A. (2015). Graphicacy in biology textbooks: Analysis of activities with images. Infancia y Aprendizaje, 1-33. DOI: 10.1080/02103702.2015.1054667
Pozzer, L. L, & Roth, W. M. (2003). Prevalence, function, and structure of photographs in high school biology textbooks. Journal of Research in Science Teaching, 40(10), 1089-1114. https://doi.org/10.1002/tea.10122
Roberts, K. L., Norman, R. R., Duke, N. K., Morsink, P., Martin, N. M., & Knight, J. A. (2013). Diagrams, timelines, and tables—Oh, my! Fostering graphical literacy. Reading Teacher, 67, 12–24.
Scherer, D., Verkühlen, A. & Dutke, S. (2023). Effects of related decorative pictures on learning and metacognition. Instructional Science, 51, 571–594. https://doi.org/10.1007/s11251-023-09618-8
Schleppegrell, M. (2004). The Language of Schooling. A Functional Linguistics Perspective. Lawrence Erlbaum.
Schnotz, W., Ludewig, U., Ullrich, M., Horz, H., McElvany, N., & Baumert, J. (2014). Strategy shifts during learning from texts and pictures. Journal of Educational Psychology, 106(4), 974–989. https://doi.org/10.1037/a0037054
Shanahan, T., & Shanahan, C. (2008). Teaching disciplinary literacy to adolescents: Rethinking content-area literacy. Harvard Educational Review, 78(1), 40–59.
Shanahan, T., & Shanahan, C. (2012). What is disciplinary literacy and why does it matter? Topics in Language Disorders, 32(1), 7–18.
Shin, W. (2017). Analysis of elementary school students’ visual attention on the editorial design of ‘structure and function of our body’ in the 2007-2009 revised elementary science textbook. Journal of Korean Elementary Science Education, 36(4), 428-438.
Šimik, O. (2021). Analysis of Visual Components in Czech History Textbooks for Lower Grades of Elementary Schools. In P. Bagoly-Simó &
Z. Sikorová (Eds.), IARTEM 2015 Textbooks and Educational Media: Perspectives from Subject Education (pp. 119–133). Springer. https://doi.org/10.1007/978-3-030-80346-9_10
Slough, S. W., McTigue, E. M., Kim, S., & Jennings, S. K. (2010). Science Textbooks’ Use of Graphical Representation: A Descriptive Analysis of Four Sixth Grade Science Texts. Reading Psychology, 31(3), 301-325. https://doi.org/10.1080/02702710903256502
van Leeuwen, T. (2005). Introducing Social Semiotics. Routledge.
Wright, K.L., McTigue, E.M., Eslami, Z.R., University, T.A., & Reynolds, D. (2014). More than just eye-catching: Evaluating graphic quality in middle school English language learners’ science textbooks. Journal of Curriculum and Instruction, 8, 89–109. https://doi.org/:10.3776/joc-i.2014.v8n2p89-109
Downloads
Published
How to Cite
Issue
Section
License
Forfatterne og Læremiddel.dk.
Må ikke bruges kommercielt.
