Transcribing Facial Gestures
Combining Jefferson with the International SignWriting Alphabet (ISWA)
DOI:
https://doi.org/10.7146/si.v6i3.143071Keywords:
multimodal transcription, facial gestures, ISWA, Sutton SignWritingAbstract
Within multimodal interaction analysis, transcripts serve not only as a tool for managing the volatility of interaction and catalyzing analytic procedures, but also as an essential medium for making analytic results intersubjectively available. While there are already well-established conventions for transcribing verbal and vocal interactional resources, researchers still struggle with adequately aligning and recognizably representing visual-bodily resources. This contribution provides a practical solution for multimodal transcription, combining conventions of the Jeffersonian system with the sign inventory of the International SignWriting Alphabet (ISWA). The result is a standardized, data oriented, expandable system that relies on iconic depictions rather than on verbal descriptions of visual conduct.
References
Aaarsand, P. & Sparrman, A. (2021). Visual transcriptions as social-technical assemblages. Visual Communication. 20(2). 289-309.
Albert, S. et al. (2019). Drawing as transcription: How do graphical techniques inform interaction analysis? Social Interaction. Video-Based Studies oh Human Sociality. 2(1). https://tidsskrift.dk/socialinteraction/article/view/113145/161800 (21.04.2022)
Ayaß, R. (2015). Doing data: The status of transcripts in Conversation Analysis. Discourse Studies. 17(5). 505-528.
Bezemer, J. (2014). Multimodal transcription. A case study. In: Norris, S. et al. (Eds.), Interactions, Images, Texts. A reader in multimodality. Berlin, Boston: deGruyter. 155-169.
Bezemer, J. & Mavers, D. (2011). Multimodal transcription as academic practice. A social semiotic perspective. International Journal of Social Research Methodology. 14(3). 191-206.
Birdwhistell, R. L. (1973). Kinesics and Context: Essays on body-motion communication. Middlesex: Penguin.
Bohle, U. (2013). Approaching notation, coding, and analysis from a conversational analysis point of view. In: Müller, C. et al. (Eds.), Body – Language – Communication: An International Handbook on Multimodality in Human Interaction (HSK 38.1). Berlin u.a: deGruyter. 992-1007.
De Stefani, E. (2022). On Gestalts and their Analytical Corollaries: A Commentary to the Special Issue. Social Interaction. Video-Based Studies of Human Sociality. 5(1). Online: https://tidsskrift.dk/socialinteraction/article/view/130875/177121 (10.05.2022).
Dix, C. (2021). Die christliche Predigt im 21. Jahrhundert: Multimodale Analyse einer Kommunikativen Gattung. Wiesbaden: Springer VS.
Dix, C. (2022). GAT 2 trifft das International SignWriting Alphabet (ISWA): Ein neues System für die Transkription von Multimodalität. In: Schwarze, C. & Grawunder, S. (Eds.), Transkription und Annotation gesprochener Sprache und multimodaler Interaktion: Konzepte, Probleme, Lösungen. Tübingen: Narr. 103-131.
Drew, P. (2017). Gail Jefferson and the development of transcription. In: Hepburn, A. & Bolden, G. (Eds.), Transcribing for social research. Los Angeles: Sage. 5-6.
Ekman, P. (1979). About brows. Emotional and conversational signals. In: Cranach, M. von, et al. (Eds.), Human ethology. Claims and limits of a new discipline. Cambridge University Press, 169-202.
Ekman, P. & Friesen, W. & Hager, J. C. (2002). Facial Action Coding System: An ebook for pdf readers. Douglas/Arizona: a Human Face.
Golato, A. (2012). German oh: Marking an Emotional Change of State. Research on Language and Social Interaction. 45(3). 245-268.
Goodwin, C. (1981). Conversational Organization. Interaction between speakers and hearers. New York (u.a.): Academic Press.
Goodwin, C. (2000): Practices of Seeing. Visual Analysis: An Ethnomethodological Approach. In: Leeuwen, T. van & Jewitt, C. (Eds.), Handbook of Visual Analysis. London: Sage. 157-182.
Goodwin, M. H. & Goodwin, C. (1986). Gesture and coparticipation in the activity of searching for a word. Semiotica. 62(1-2). 51–75.
Heath, C. et al. (2010): Video in Qualitative Research. Analysing social interaction in everyday life. Los Angeles: Sage.
Heller, V. (2021). Embodied displays of 'doing thinking'. Epistemic and interactive functions of thinking displays in children's argumentative activities. Frontiers in Psychology. 12. 1-21.
Hepburn, A. & Bolden, G. (2017). Transcribing for social research. Los Angeles (u.a.): Sage.
Hoffmann-Dilloway, E. (2018). Feeling your own (or someone else’s) face: Writing signs from the expressive viewpoint. Language and Communication. 61. 88-101.
Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In: Lerner, G. H. (Ed.): Conversation Analysis. Studies from the first generation. Amsterdam, Philadelphia: John Benjamins. 13-31.
Jenkins, C. (2018). Recording and transcribing social interaction. In: Flick, U. (Ed.): The SAGE handbook of qualitative data collection. Sage. 118-130.
Kendon, A. (2004). Gestures. Visible Action as utterances.
Kennedy, A. (2013). Laban based analysis and notation of body movement. In: Müller, C. et al. (Eds.), Body – Language – Communication. An international Handbook on Multimodality and Human Interaction. (HSK 38.1). 941-958.
Knoblauch, H. & Heath, C. (2006). Die Workplace Studies. In: Rammert, W. & Schubert, C. (Eds.), Technografie. Zur Mikrosoziologie der Technik. Frankfurt, New York: Campus. 141-161.
Laurier, E. (2014). The Graphic Transcript: Poaching comic book grammar for inscribing the visual, spatial and temporal aspects of action. Geography Compass. 8(4). 235-248.
Luff, P. & Heath, C. (2015). Transcribing embodied action. In: Tannen, D. et al. (Eds.): The Handbook of Discourse Analysis. John Wiley. 367-390.
Mittelberg, I. (2007). Methodology for multimodality. One way of working with speech and gesture data. In: Gonzalez-Marquez, M. et al. (Eds..), Methods in Cognitive Linguistics. Amsterdam. Philadelphia: John Benjamins. 225-248.
Mondada, L. (2007). Commentary: Transcript variations and the indexicality of transcribing practices. Discourse Studies. 9(6). 809-821.
Mondada, L. (2017). Multimodal transcription and the challenges of representing time. In: Hepburn, A. & Bolden, G. (Eds.), Transcribing for social research. Los Angeles: Sage.
Mondada, L. (2018). Multiple Temporalities of Language and Body in Interaction. Challenges for Transcribing Multimodality. Research on Language and Social Interaction. 51(1). 85-106.
Mondada, L. (2019a). Conventions for multimodal transcription. Online: https://www.lorenzamondada.net/multimodal-transcription (10.05.2022).
Mondada, L. (2019b). Transcribing silent actions: A multimodal approach of sequence organization. Social Interaction. Video-Based Studies of Human Sociality. 2(1). Online: https://tidsskrift.dk/socialinteraction/article/view/113150 (10.05.2022)
Parkhurst, S. & Parkhurst D. (2010). A cross-linguistic guide to SignWriting. A phonetic approach. Online: http://www.signwriting.org/archive/docs7/sw0617_Cross_Linguistic_Guide_SignWriting_Parkhurst.pdf (21.04.2022).
Rossano, F. (2013). Gaze in Conversation. In: Sidnell, J. & Stivers, T. (Eds.), The Handbook of Conversation Analysis. Oxford: Blackwell. 308-329.
Sager, S. (2001). Probleme der Transkription nonverbalen Verhalten. In: Brinker, K. et al. (Eds.), Text- und Gesprächslinguistik. Ein internationales Handbuch zeitgenössischer Forschung. (HSK 16.2). 1069-1085.
Selting, M. et al. (2009). Gesprächsanalytisches Transkriptionssystem 2 (GAT2). Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion 10, 353-390. Retrievable online: http://www.gespraechsforschung-online.de/heft2009/heft2009.html (retrieved 30th Oct. 2020).
Streeck, J. (2009). Gesturecraft: The manufacture of meaning. Amsterdam, Philadelphia: John Benjamins.
Stukenbrock, A. (2009). Herausforderungen der multimodalen Transkription: Methodische und theoretische Überlegungen aus der wissenschaftlichen Praxis. In: Birkner, K. & Stukenbrock, A. (Eds), Die Arbeit mit Transkripten in Fortbildung, Lehre und Forschung. Mannheim: Verlag für Gesprächsforschung. 144-169.Online abrufbar: http://www.verlag-gespraechsforschung.de/2009/birkner.html (30.10.2020).
Sutton, V. (2010). The SignWriting Alphabet. Read and Write any Sign Language in the World. ISWA Manual 2010. The SignWriting Press. http://www.movementwriting.org/symbolbank/ (30.10.2020).
Sutton, V. (1981/1982). Sutton Movement Writing and Shorthand. Dance Research Journal. 14(1 & 2). 78-85.
Wagner, J. (2020). Conversation Analysis: Transcriptions and Data. In: Chapelle, C. (Ed.), The concise encyclopedia of applied linguistics. Wiley-Blackwell. 296-303.
Waller, B. M. & Smith Pasqualini, M. (2013). Analysing facial expression using the facial action coding system (FACS). In: Müller, C. et al. (Eds.): Body – Language – Communication: An International Handbook on Multimodality in Human Interaction (HSK 38.1). Berlin u.a: deGruyter. 917-931.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Author and Journal
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
We follow the Budapest Open Access Initiative's definition of Open Access.
The journal allows the author(s) to hold the copyright without restrictions.
The journal allows software/spiders to automatically crawl the journal content (also known as text mining)
The journal provides article level metadata to DOAJ
The journal allows readers to read, download, distribute, print, search, or link to the full texts of its articles and allow readers to use them for any other lawful purpose.
