Social Interaction.
Video-Based Studies of Human Sociality.
2019 VOL. 2, Issue 1
ISBN: 2446-3620
DOI: 10.7146/si.v2i1.113968
Social Interaction
Video-Based Studies of Human Sociality
The panel show: further experiments with graphic transcripts and vignettes
Eric Laurier,
Institute of Geography & the Lived Environment, University of Edinburgh, UK
Orcid: 0000-0001-8043-0710
Scopus: 6701537810
Abstract
The analysis of video recordings in EMCA has brought with it an attentiveness to the embodied, mobile, visual, ecological and other aspects of members’ practices. In this article, the use of comic strips comics to transcribe video, and to construct vignettes that exhibit video materials as comic strips, are explored as a response to EMCA’s concern about losing the phenomena and participants’ perspectives. Two graphic transcripts and two graphic vignettes, each based on the same recording, are presented in order to consider which aspects of practices they assist the researcher in exhibiting.
Keywords: ethnomethodology; conversation analysis; transcription; comics; graphic transcripts; graphic vignettes
Introduction
Recordings of members’ practices provide the materials for the beginnings of EMCA inquiries into practical reasoning, accountability, sequence organization, categorizations and other topics. From their outset they have also inspired creative and initially, at least, unconventional ways of rendering those practices to avoid losing the phenomenon. In this short article I aim to document further experiments with comic strips as a medium for exhibiting phenomena from video recordings. In the first part, I will examine two variations on transcribing the same event (two café customers preparing to pay) and, in the second, two variations on creating a vignette of the same recording which then foreground different analytic issues. The first vignette describes a practice by which café staff progress payment, the second describes the problems of café customers in locating whom to pay.
The Graphic Transcript
The writing or typing of transcripts is itself a form of proto-theorizing and the confluence between a concern with theorizing turn-taking in talk and the turns generated by the line-by-line nature of transcript has been remarked on many times before (Gibson, Webb, & vom Lehn, 2014; Mondada, 2018; Ochs, 1979). While the textual transcript continues to have fruitful convergences with turn-by-turn analysis of talk, it has fitted less well with the continuous parallel gestalts of embodied action in ecologies of objects and architectures which interactional analysts come across in video recordings (Mondada, 2018). The many courses of action witnessed in the video recording, actions that are unfolding alongside and interwoven with each other, lead to intricate textual production processes and equally complex reading practices.
Part of learning conversation analysis is learning to write and read its transcripts, distinguished by the audible features that are marked by what are usually called Jeffersonian conventions. These textual markers register otherwise lost, audible qualities of talk such as pace, loudness, laughter particles, inbreaths and so on (Hepburn & Bolden, 2017). In providing resources for studying the organization of talk through the turn-taking system, one line ends, or should end, where the transcriber has found a potential point for an acceptable transition from one speaker to another. Sometimes the lines overlap with one another, registering where two or more parties are talking at the same time. Absence of speech is marked by brackets with numbers.
The Jeffersonian conventions were designed not just to help scholars register and notice conversational features when listening to audio recordings, but also to converge with the features of talk that speakers themselves use to analyse the emerging conversation. When EMCA researchers began transcribing video recordings, the Jeffersonian system was taken to be the place to start and then to further augment and elaborate it in order to capture non-verbal features of members’ practices. Most conspicuously, additional parallel lines of action accompany talk. These parallel lines track features such as gaze, gestures, the manipulation of objects, walking, dance steps, typing at keyboards and many other potentially recognizable and relevant courses of action. The most commonly used system for transcribing video recordings is the one that has been developed by Mondada (2014). Brackets, with time in seconds, which previously registered the absence of talk, are used in the Mondada system to inscribe measures of continuity, of starting and stopping (Luff & Heath, 2015; Mondada, 2014).
Building on Jefferson’s elegant solution, the Mondada conventions continue to use the length of the transcribed text as a rough measuring stick for the timing of actions produced. The lines of text become in themselves timelines. Timing is foregrounded because the text seems to show, with a degree of precision, what starts first, what finishes last and what runs in parallel. This precision is sometimes, though certainly not always, a members’ resource. Having outlined elements of the qualities of the textual transcript, I would like to turn to the graphic transcript (Laurier, 2014).
The graphic transcript has cross-cutting temporalities because it is an adaptation of the comic strip. The comic strip, made up of sequences of panels, images, speech bubbles and captions, has particular properties for the transcribing of a recording, and for the reading practices that animate it. Even as the panel-to-panel progression of the strip runs forward, there is reading to be done, and sense to be found, in each panel. The strip is read across the gutters, by pairing the panels into progressive sequences, yet also from readers glancing around the larger page of panels (Doel, 2014; Gallacher, 2011). Each panel from the strip has lines of speech within it, although usually they are inside bubbles and so the speech splatters and pops instead of running along tracks as the line of text does. The strip of panels potentially has other text lines to be read in the caption boxes, captions that are then read in relation to the panel and are routinely used to establish tense or shape a sense of time. The image itself may establish a sense of duration, representing anything from a millisecond to a million years (McCloud 2006; Grennan 2017). The strip of panels forms lines, akin to lines of text, yet the panels also form grids. Where, then, the multiplicity of temporalities at work is an unexpected quality in the textual transcript, the multiplicity of durations and temporalities is both a problem and a possibility for the graphic transcript.
The properties of panels as part of strips
While graphic transcripts lack the simplicity with which textual transcripts’ indicate duration and timing, they bring an array of other indications of the locally observable, reportable and accountable features of what is happening. The experiments that I will document in this section were produced as part of a collaborative research project (https://sites.eca.ed.ac.uk/comicpraxis/) between myself and an illustrator. We worked through several variations on graphic transcripts on the basis of undertaking data sessions together. The particular fragment, in this example, is of two customers preparing to pay while approaching the counter. I have picked out two of the formats that respond to the problems of timing produced by the graphic transcript. The first experiment provides a relatively simple convention, used already in comic strips, of using panel and gutter width to show duration. In our experiment we foregrounded this quality by cropping the images to allow us to more easily produce contrasting lengths of panels which then also increases the emphasis on a progressive left-to-right reading. The latter maintains the sense of temporal sequencing in contrast to reading practices that might traverse the panels in multiple directions (see Doel (2016) on the non-sequential comic).
Figure 1. Panel width, gutter gap and bubble overlap used to index duration.
Let’s begin by focusing on the top row of panels and examining with the third panel of figure 1, where Customer 1 (C1) shows her ticket to Customer 2 (C2). The panel’s narrowness, which indicates its brevity, is established through the majority of panels in the strip being of a similar length and it is in relation to that established rhythm that the narrow panel can be seen at a glance, as a brief moment. The speech bubble shows the timing of C1 displaying the ticket whilst at the same time saying ‘I haven’t lost my ticket’. It has its tail in the previous panel, its bubble sits across the gutter of the ticket panel and reaches to the next panel, where C1 and C2 return to looking at their purses.
Each ‘frozen’ image is seen as standing for courses of action or as ‘proto-narratives’ (Jayyusi, 2004) and is then itself in the witnessable order of a narrative strip with the preceding and succeeding images. In the second panel, just before showing the ticket in the third panel, the image is of C1 turning toward C2 when she begins to speak, after which, when she shows the ticket, C1 is still turned towards C2. C2 has not reciprocated and is instead investigating the contents of her own purse. In the fourth panel, on which the speech bubble is also overlaid, C1 is no longer turned towards C2 and has shifted back to her own purse. The video frames are selected to show the turn as sequenced and timed, the speech beginning ahead of the appearance of the ticket and showing C2’s disattention in the narrow frame.
Turning to the bottom row of panels in figure 1, in the middle of the strip the gutter is narrower than the usual gutter pattern and this is combined with a narrowing of the panels. By this use of gutter and panels, the pairing of ‘will we make it fifteen’ and ‘uh huh’ is then available as closely timed compared to the customer’s utterances before and afterwards. Moreover, we used the stretching of bubbles across the gutters to help understand the flow and timing of the frames.
In the second example from the experimental work, the strips of panels are laid out between the line-by-line arrangement of textual transcripts, thereby blending textual transcript and comic strip. Figure 2 is from the same part of the recording as the first two rows of figure 1. From one transcribing experiment to the next, the illustrator had become curious about C2’s absence of response to being shown the ticket. When we listened to the clip again before making the next transcript, we were able to discern that C2 was responding verbally to C1 (e.g. ‘oh good’) even though she didn’t meet her gaze. The audio is hard to hear when the customers are at a distance from the microphone (which is located on the counter).
Figure 2. Hybrid panel and transcript text arrangement.
In producing the transcript in this way the convention of placing captions in the corner of the panel was also then opened up for alteration. The captions were closer equivalents of the Mondada (2014) conventions for marking lines of embodied action in parallel to lines of talk. Running the text along the edges of strip, however, allowed us to remove the brackets representing time length of actions when there is no speech (e.g. as in fig. 3).
Fig. 3 Time brackets as used in a multi-modal transcript
The picture strips and speech bubbles do not have the tenth-of-a-second precision of the textual transcript, yet they provide for the recognizability of orderly, observable and relevant features for members. It is in the refashioning of them for EMCA purposes that we rediscover rendering as a method for finding overlooked things. Timing and indeed tenths of a second time, is itself not always foregrounded and instead only occasionally provides the focus. As Bogen (1999) argued, Jeffersonian transcripts in CA publications work in a literary pairing with the accompanying instructed reading from the author, directing the reader as to what they should find in the transcript. The transcript itself, as Bogen argues, then creates a new temporal order by presenting the whole course of the event[i] that is available to the reader and that was not available in the original event. The reader then, in the subsequent analysis, has a potential retrospective reading that inquires into how the preceding sequences of action led up to the current one. The path-to-the-final-outcome analysis of transcripts is hard to avoid and the discipline of the EMCA scholar is in recovering the eventfulness and sense of the moment that only projects. One promising aspect of the graphic transcript is that it might not require the same pairing in order to understand the events that it is presenting, even if it still suffers from providing the reader with the events that follow the current panel.
The graphic vignette - Captions, narrative and perspective shift
The rise of the ‘graphic novel’ in English language comics has marked a shift in ambition toward the genres, criteria and history of the novel, and it has expanded beyond the novel into a number of other forms of writing: autobiography; biography; history; foreign reporting; travel writing; popular science; philosophy; key ideas in critical theory; PhD theses (Sousanis, 2015). One of the scholarly of comics (McCloud (2006) which I use in this article and elsewhere) is in the form of a comic strip. Beyond transcribing video recordings, the comic strip offers EMCA investigations a different form for presenting those investigations. In pursuing this possibility, I began creating graphic vignettes. The vignette is a standard form in ethnographic fieldwork for beginning to analyse events from fieldwork which can be based on written records, photographic records and, of course, video records. The graphic vignette emerges from a video recording and is usually a step further in analytic terms the proto-analysis of the transcripts.
In the continuing collaboration with comic strip artists, I and several other researchers worked with an experienced producer of and scholar of comics (Grennan, 2017)[ii]. As part of this experiment we adapted the use of captions in order to expand their analytic work. In the previous section on transcripts, captions accompanied the image, replicating the function of the double-bracketed captions in Jefferson transcripts. In order to produce a vignette, an unboxed caption which contained a textual narrative was placed below or above the panels (see figures 4 and 5). The size of the panels were altered, this time as part of practices showing and widening a spatial field of focus rather than displaying the duration of each panel. The video frames were cropped in relation to the same objective. The literary pairing between the transcript and the analyst’s instructed reading of the transcript was reshaped in the process: there was neither a separate transcript, nor an instructed reading of that transcript. In short, the comic strip absorbed the surrounding text to become a graphic vignette.
Where the genesis of the Jeffersonian conventions brought an abiding concern with turn-taking, in the graphic vignette we brought to its development a concern with the concept of perspective. Perspective is a feature of the recording process that has already been explored in EMCA reflections on recording (Luff & Heath, 2012). It also appears in audio recording, where its significance is often missed[iii]. Given the graphic vignette’s basis in the video record, perspective provided an additional challenge because the video record was from a lone camera[iv]. The aim, in creating the graphic vignette, was to shift perspective without shifting camera perspective and, as we shall see, there was a graphic machinery for addressing the idea.
Fig 4. Graphic Vignette – payment practices at the counter. Shifting from staff to customer perspectives through following the bill
A first thing to notice, in figures 4 and 5, is that the amount of words the maker of a graphic vignette can use are limited. Because the narrative text is placed amongst the strip of panels, were it to be too extensive it would fail to maintain the panel-to-panel progression that makes reading possible for a comic strip reader. The text would simply overwhelm the panels and the gestalt would tend instead toward an illustrated text.
As described earlier, the restriction of the camera’s angle, its distance from and perspective on its subjects seems to set limits on what is shown. Yet even with the fixed camera shot, the cropping of the frame can be used to shift the intended focus of attention. In figure 4, the first frame is cropped widely to show both staff and customers, its caption brings the customers into focus, even though they are at the rear of the frame. In the next frame, however, the action is not focused on them. In the developing sense of the strip, it is the staff that we focus on and indeed, in the third frame, the customers are cropped out entirely. However, the large-sized and wide-cropped panel pulls the reader back, taking in the overall space of the café, and returning the reader’s attention to a situation that includes staff and customers. The shift in perspective is also driven by one of the staff turning toward the customers, even as she continues to talk to the other member of staff. As the comic strip continues, I also shift the reader’s focus both back toward the customers’ response and the role of the objects and the furnishings (the movements of the bill and the saucer on the counter). My experiments in the production of perspectives via the comic form then helped me as an analyst to consider participants’ understandings and production of the situation. In the case of developing the vignette in figure 5, which I will not comment on, it helped me distinguish the staff’s problems from the customers’ problems.
Fig 5. Graphic Vignette – Customer perspective with paired frames
In figure 5, as mentioned earlier, the vignette is of the same video fragment. By drawing upon the machinery of the comic strip I shift the reader’s attention toward the customers’ perspective and the customers’ problems. By using cropping, captioning and the accompanying narrative there is, then, a shift in analytic concern without the need for two point-of-view cameras. In the first pair we are shown the two customers and, rather than use an annotation to circle them, they are cropped in order to guide our attention to them. The framing shows them as single customers before they then constitute a ‘together’ in the café. We can note that there is potential confusion in reading the first frame because there is a customer in the foreground who is also in better focus. Our target customer solidifies as our reading of the panels progresses into the next panels and may trigger a re-inspection of the first panel to find our target customer present, but more blurred, in the background.
Returning to EMCA’s concern with the timing of actions and events, the perception of when things happen certainly need not be lost, even though in the vignettes timing is no longer constantly visible as a resource for the reader. In figure 5, cropped parts of the wider video shot are paired by thick black gutters to show temporal co-occurrence. The paired frames are used next to allow us to see what one customer looks at (the staff at the counter) and then, in the next pair, to show us what both customers look at that point in time (another waitress that is still moving amongst the tables). In the final pairing we see the customers again and the thing that they see that leads them to approach the counter. In short, through cropping and then pairing, with an EMCA concern with participation and the embodied organization of joint action, we can consider the actions of the participants in the situation and examine them in relation to the co-occurrence of other events and actions.
What next?
EMCA investigations into comics profitably pursue a deeper understanding of the practical reasoning of comic strip production through producing comics. A set of inquiries based on researchers making comics can then produce an account comparable to those documented by EMCA in scientific demonstrations (Bjeliç 2003), one that could render the appearance of EMCA studies as themselves comic.
What I have begun to do by producing vignettes is to reconfigure the relationship that video data has to the comic strip from one of reproducing the video record as closely as possible to one of providing an EMCA account of the event where the video is an aid to recovering the eventfulness of the event and the lessons that we can learn from members’ practices.
Acknowledgements
Ryan Hamill, Simon Grennan, Tim Smith, Shari Sabeti & Harvey Dingwall and other members of the Comics Praxis Group at the University of Edinburgh for making Comics as Research happen and bringing so many amateur comic producers together.
References
Ashmore, M. (2000), “Innocence and nostalgia in conversation analysis: The dynamic relations of tape and transcript”, FQS: Forum Qualitative Sozialforschung/ Qualitative Social Research, Vol. 1 No. 3.
Bjelić, D. I. (2003), Galileo’s pendulum: science, sexuality, and the body-instrument link, State University of New York Press, New York.
Bogen, D. (1999), Order Without Rules, SUNY Press, New York.
Broth, M., Laurier, E. & Mondada, L. (2014), “Introducing Video at Work”, in M. Broth, E. Laurier, & L. Mondada (Eds.), Studies of Video Practices, Routledge, New York, pp. 1-29.
Doel, M. A. (2014), “And So. Some Comic Theory Courtesy of Chris Ware and Gilles Deleuze, Amongst Others or, an Explication of Why Comics Is Not a Sequential Art”. in J. Dittmer (Ed.), Comic Book Geographies, Franz Steiner Verlag, Stuttgart, pp. 1-29.
Gallacher, L. A. (2011), “(Fullmetal) alchemy: the monstrosity of reading words and pictures in shonen manga”, Cultural Geographies, Vol. 18 No. 4, pp. 457–473.
Garfinkel, H. (2002), Ethnomethodology's Program, in A. W. Rawls (Ed.), Rowman & Littlefield, London.
Gibson, W., Webb, H. & Lehn, vom, D, (2014), “Analytic Affordance: Transcripts as Conventionalised Systems in Discourse Studies”, Sociology, Vol. 48 No 4, pp. 780–794. http://doi.org/10.1177/0038038514532876
Grennan, S. (2017), A Theory of Narrative Drawing, Springer, New York.
Hepburn, A. & Bolden, G. B. (2017), Transcribing for Social Research, Sage, London.
Ivarsson, J. (2010), “Developing the construction sight: Architectural education and technological change”, Visual Communication, Vol. 9 No. 2, pp.171–191.
Jayyusi, L. (2004), “The reflexive nexus: photo-practice and natural history”. Continuum: the Australian Journal of Media & Culture.
Jefferson, G. (1978), “What's in a `Nyem'?” Sociology, Vol. 12 No. 1, pp. 135–139. http://doi.org/10.1177/003803857801200109
Laurier, E, (2014), “The Graphic Transcript: Poaching Comic Book Grammar for Inscribing the Visual, Spatial and Temporal Aspects of Action”, Geography Compass, Vol. 8 No. 4, pp. 235–248. http://doi.org/10.1111/gec3.12123
Luff, P. & Heath, C. (2012), “Some “technical challenges” of video analysis: social actions, objects, material realities and the problems of perspective”, Qualitative Research, Vol. 12 No. 3, pp. 255–279. http://doi.org/10.1177/1468794112436655
Luff, P. & Heath, C. (2015), “Transcribing Embodied Action”, in D. Tannen, H. E. Hamilton, & D. Schiffrin (Eds.), The Handbook of Discourse Analysis, pp. 397–321, Wiley Blackwell, Oxford.
McCloud, S. (2006), Making Comics, Harper, London.
Mondada, L. (2014), Conventions for multimodal transcription (pp. https:––mainly.sciencesconf.org–conference–mainly–pages–Mondada2013_conv_multimodality_copie.pdf), Retrieved from https://mainly.sciencesconf.org/conference/mainly/pages/Mondada2013_conv_multimodality_copie.pdf
Mondada, L. (2018), “Multiple Temporalities of Language and Body in Interaction: Challenges for Transcribing Multimodality”, Research on Language and Social Interaction, Vol. 51 No. 1, pp. 85–106. http://doi.org/10.1080/08351813.2018.1413878
Ochs, E. (1979), “Transcription as theory”, in E. Ochs & B. Schieffelin (Eds.), Developmental Pragmatics, Academic Press, New York, pp. 43-72.
Sacks, H. (1992), Lectures On Conversation, Blackwell, Oxford.
Sharrock, W. W. & Anderson, R. J. (2017), “Has Ethnomethodology Run Its Course?”, University of Manchester, Manchester.
Sousanis, N. (2015), Unflattening, Harvard University Press, Harvard.
Stokoe, E. H. (2014), “The Conversation Analytic Role-play Method (CARM): A Method for Training Communication Skills as an Alternative to Simulated Role-play”, Research on Language and Social Interaction, Vol. 47 No. 3, pp. 255–265. http://doi.org/10.1080/08351813.2014.925663