Editorial

 

Special Issue: Representing data in video-based studies

This journal publishes studies that investigate social interaction by using video-based and documentary methods of representing data typically by annotating and transcribing it. Many people have been trained in transcription systems such as Jefferson (2004) or GAT (Selting et al., 1998), which works very well by itself when analysing telephone calls (Sacks, Schegloff, & Jefferson, 1974), but we also encounter numerous challenges when trying to collect and represent the vivid and complex situations of embodied multimodal interaction in spatial environments populated by objects and material structure.

 

The embodied nature of face-to-face or technology-mediated interaction makes it even more evident that the whole empirical process, from the first encounter with people on a setting, choosing types, amounts and positions of cameras to systems and traditions of working with raw data and transcriptions, contains numerous choices and subtle interpretations along the way (Heath, Hindmarsh, & Luff, 2010; Due, 2017). Capturing the complexity of locally emerging semiotic fields (Goodwin, 2017, 2018) and the gestalts of multimodal resources (Mondada, 2014) poses many new questions about, for example, what embodied actions to include in the transcription and how.

 

There is evidently no such thing as a perfect representation of that locally emerging event, the haecceity, the simple thisness (Garfinkel, 1991), of that particular moment. It is a premise that something is always missing (Ochs, 1979). But the aspiration has nevertheless always been to get closer to the “reality” by using new technologies, systems and methods for representation. Obviously, a slice of positivism is inherent in this aspiration. However, no matter what kind of sophisticated technological setup we can apply and no matter how accurate the annotation system gets, we are always dealing with interpretations.

 

Studies based on ethnomethodology and conversation analysis establish arguments grounded in detailed analysis of shown data displayed as various forms of transcriptions. When presented in a paper, we have no way of checking the soundness of that argument other than by examining the analysis of the data as it is represented. Thus, annotating and representing data in a paper is vital for the reliability and validity of these kinds of studies.

 

One of the initial aims of the journal was closely related to issues of representing the empirical video-based material. Video recordings of naturally occurring interaction have become the default methods for recording face-to-face interaction, whether collected by ourselves as part of a video ethnographic study or harvested from, for example, YouTube, and the ability to make these video recordings part of a publication seemed important. While some scholars use this feature in this journal if it suits their argument and if the video quality is good enough and participants have accepted it and it is anonymized, others choose for the same reasons to leave out the videos.

 

Five remarkable contributions in this special issue

It is therefore with great pleasure that we publish this special issue on the topic with contributions from scholars working within the broad spectrum of EMCA.

 

The paper by Paul McIlvenny introduces and discusses a complex virtual annotation system based on 360-degree recordings and the use of VR glasses. In his paper: “Inhabiting spatial video and audio data: Towards a scenographic turn in the analysis of social interaction” McIlvenny discusses the possibilities and restrictions with such a system. Whereas Word or transcription systems such as Elan, Clan or Transana are commonly used software for transcription, McIlvenny introduces what we might call a paradigm shift in the practices of annotating. The system provides the ability to “walk into” the recordings and annotate the video material directly. Read McIlvenny’s paper to get a clearer understanding on how this system works. In this special issue, McIlvenny’s paper represents the first phases of the empirical process: collecting and annotating the raw material.

 

The remaining papers are more concerned with the practice of transcribing and how different types of representations inform the analysis. Four different papers introduce and discuss different transcription systems.

 

The paper by Saul Albert, Claude Heath, Sophie Skach, Matthew Tobias Harris, Madeline Miller and Patrick G. T. Healey is entitled “Drawing as transcription: how do graphical techniques inform interaction analysis?” This paper investigates the possibilities of enhancing the understanding of embodied actions by using drawing techniques in the annotation process. Their argument is that drawing as a form of analytical inscription can provide researchers with highly flexible methods for exploring embodied interaction. They demonstrate how graphical techniques can be used in interaction research by illustrating the postural configurations and movements of participants in a ballet class and they introduce a prototype software tool that is being developed to support interaction analysis.

 

Eric Laurier is also concerned with the visual and graphical aspects of transcriptions, but in a completely different way. In his paper: The panel show: further experiments with graphic transcripts and vignettes, Laurier introduces the use of comic strips to transcribe video, and to construct vignettes that exhibit video materials as comic strips. Two graphic transcripts and two graphic vignettes, each based on the same recording, are presented in order to consider which aspects of these practices they help the researcher to exhibit.

 

In their paper: Some theoretical and methodological challenges of transcribing touch in talk-in-interaction Luca Greco, Renata Galatolo, Anne Sylvie Horlacher, Vanessa Piccoli, Anna Claudia Ticca and Biagio Ursi deal with theoretical and analytical issues raised by the transcription of touching practices. They discuss among other things how to transcribe purely visual and tactile actions.

 

Finally, this special issue also contains a contribution from Lorenza Mondada on “Transcribing silent actions: a multimodal approach of sequence organization”. As Mondada’s system and symbols for transcribing bodily conduct has become increasingly widespread, questions have also begun to arise concerning, for example, how to transcribe silent actions. She argues that close attention to how we transcribe silent embodied actions enables us to better understand their specific temporal unfolding, spatial arrangements and sequential organization. Her proposed transcript notation enables reflection upon the complex emergent and sequentially unfolding temporality of multimodally formatted actions.

 

Overall, we believe that these papers both represent some of the current topics in video-based interactional studies concerning representation of the empirical world and at the same time present us with new, detailed and analytically-based findings on human sociality.

 

The editors,

Brian Due & Kristian Mortensen

 

 

References

 

Due, B. (2017), Multimodal interaktionsanalyse og videoetnografisk dataindsamling. Samfundslitteratur.

Garfinkel, H. (1991), "Respecification: evidence for locally produced, naturally accountable phenomena of order, logic, reason, meaning, methods, etc. in and of the essential haecceity of immortal ordinary society (I) - an announcement of studies”, in I G. Button (Ed.), Ethnomethodology and the Human Sciences , Cambridge University Press, Cambridge, pp. 10-19.

Goodwin, C. (2017), Co-Operative Action, Cambridge University Press.

Goodwin, C. (2018), "Why Multimodality? Why co-operative action? Social Interaction. Video-Based Studies of Human Sociality", Vol. 1 No. 2.

Heath, C., Hindmarsh, J., & Luff, P. (2010), Video in Qualitative Research, SAGE Publications Ltd.

Jefferson, G. (2004), "Glossary of transcript symbols with an introduction", in Gene H. Lerner (Ed.), Conversation Analysis: Studies from the first generation, John Benjamins Publishing Co, pp. 13-31.

Mondada, L. (2014), "The local constitution of multimodal resources for social interaction", Journal of Pragmatics, Vol. 65, pp. 137–156. https://doi.org/10.1016/j.pragma.2014.04.004

Ochs, E. (1979), "Transcription as Theory", in E. Ochs and B. Schieffelin (Eds.) Developmental Pragmatics, Academic Press.

Sacks, H. L., Schegloff, E. A. & Jefferson, G. (1974), "A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language", Vol. 50No. 4, pp. 696–735.

Selting, Margret, Auer, Peter, Barden, Birgit, Bergmann, Jörg, Couper-Kuhlen, Elizabeth, Günther, Susanne, . . . Uhmann, Susanne. (1998), "Gesprächsanalytisches Transkriptionssystem (GAT). Linguistische Berichte", Vol. 173, pp. 91-122.