Social Interaction

2. Data and methods

The data consist of video-recordings of 11 hospital encounters with video-mediated interpreting. Three meetings were recorded from the interpreter’s location, one from the hospital facilities, and seven from both the hospital facilities and the interpreter’s location. The analysis draws on insights from the recordings from both the hospital facilities and the interpreter’s location.

The recordings include meetings in different wards, in the form of both meetings with admitted patients who may have spent several weeks at the hospital, and brief consultations in an outpatient clinic. In most cases, the medical professionals and patients have met many times before. The medical professionals involved in these meetings have varying degrees of experience using the technology and with video-mediated interpreting. All of the interpreters in the study have formal professional qualifications in interpreting. Some of the interpreters in the study are participating in video-mediated interpreting for the first time, while some have done it many times before. With their diverse experiences and varied prior knowledge of this type of situation, the participants deduce how to accomplish video-interpreting in collaboration in situ.

The interpreter participates from a remote location – an interpreting studio equipped with a desktop system for videoconferencing. The device is about the size of a personal computer and is designed for videoconferencing from an individual workspace. It has a screen, camera, loudspeakers, microphone and a control panel. The camera has a narrow angle, and can only capture a narrow area in front of the camera. The room is also equipped with a separate telephone. In the setting illustrated above, the interpreter has a personal computer in their workspace. The interpreter usually has a pen and a notepad, and can take notes during the session.

A large screen at the end of the meeting room table serves as both a screen for the computer in the room and as a video-conferencing screen. In addition to the camera above the screen, a multidirectional microphone is connected to the system by cable and can be placed on the table. The room is furnished for video-conferencing, and the camera can capture all of the participants around the table, depending on the chosen settings. Adjustments to the video-conference system and technical settings are made using a control panel on the table next to the screen. Some wards are equipped with smaller systems, like the desktop unit in the interpreter’s studio (see illustration C).

The setting illustrated above is in a multipurpose room used both for specific types of examinations and appointments that require interpreting. A computer for access to journal systems and a desktop video-conferencing unit are available on the desk. The device is designed for video-conferencing from an individual workspace and has a narrower camera angle than the more advanced system illustrated above. Since adjustments cannot be made to the camera angle, adjustments would have to be made to seating arrangements in front of the camera in order to capture all participants at the ward.

The analysis builds on video-ethnography and multimodal conversation analysis tending to the sequential organisation of interaction within this specific linguistic and material setting. The analysis builds on the theoretical framework of multimodal conversation analysis (e.g. Deppermann, 2013; Hazel, Mortensen, & Rasmussen, 2014; Mondada, 2014).

The research project is carried out with the approval of the Norwegian Centre for Research Data, and the hospitals and wards involved. All participants have given their informed consent.

3. Analysis

This analysis demonstrates and discusses how participants-in-interaction use and orient to the media’s visual materiality and the visual interactional space or ecology afforded by the technology. The analysis’ three sections demonstrate different ways in which the visual affordance of the media becomes relevant to participants in the organisation of interpreting and the interpreted encounter. The first section demonstrates how the visual ecology informs interpreters’ work. The second demonstrates how participants use embodied resources in the organisation of interpreting within this interactional space. The final section discusses how participants orient to the visual materiality of the media.

The ongoing activities at the wards are made available to the interpreter through the use of video technology. For the interpreter to be able to interpret, he/she has to be able to hear what the participants at the ward are saying. While the interpreter’s need for access to sound from the ward may seem obvious, how visual access contributes to the interpreter’s work is perhaps less so. The following section demonstrates how the interpreter’s access to the ward serves as a source of information and informs the interpreter’s actions and linguistic choices.

Extract (1) demonstrates how the visual transmission from the hospital meeting room informs the interpreter’s work. The medical professional (MP) and researcher (RES) are present in the hospital meeting room. The meeting room is equipped with a video-conference system. The camera captures almost the entire meeting room, and the video technology transmits this signal to the interpreter, who is participating from a remote location. Shortly after contact is established between the hospital meeting room and the interpreter’s studio, the medical professional begins to inform the interpreter about the order of business. The patient (PAT) and next-of-kin (NOK) arrive as she is doing so.

Gazing at the patient, the doctor asks the patient how he is feeling (line 2–4). The Norwegian word “du” (you, second person singular pronoun) does not distinguish between genders. While the participants at the hospital meeting room can see at whom the doctor is directing his utterance, the interpreter cannot. Interpreting to Vietnamese, the interpreter has to choose an appropriate pronoun for the addressee. The patient has already responded to the doctor’s utterance in Norwegian (line 5, line 7). However, he is seated to the side of the technology, and parts of his utterance overlap with the doctor’s utterance. This makes his utterance difficult to perceive remotely. Although the patient has responded, and done so in Norwegian, the participants still treat interpreting as relevant. In her rendition, the interpreter uses the pronoun chị (big sister), a polite pronoun common for addressing female participants in interaction. The patient’s wife turns to the screen representing the interpreter (fig 2.1), the screen-interpreter, and responds in Vietnamese to the interpreter’s rendition of the question. In the form of an other-repair (Schegloff, Jefferson, & Sacks, 1977) directed through gaze and bodily orientation to the screen-interpreter, the next-of-kin makes it clear that it is her husband who is the patient, not herself (line 10–11). Although this utterance gives the interpreter information about the participants at the ward, she cannot see to whom the utterance is directed. To the interpreter, the verbal utterance is disconnected from the environment, the participation framework, in which it was produced. As such, for the interpreter, the utterance is fractured from the ecology in which it was produced (Luff et al., 2003). The interpreter proceeds to interpret the utterance into Norwegian (line 12–13). As the interpreter completes the rendition, the next-of-kin begins to laugh. The linguistic problem in Vietnamese is not available to the doctor, who only speaks Norwegian. The interpreter’s utterance is designed as a rendition of what the next-of-kin said. To the doctor, the interpreter’s rendition appears to be directed to him, which in turn occasions his response – he tells the next-of-kin that he knows this and begins to laugh. While the next-of-kin’s utterance was visually directed to the screen-interpreter, she turns back to the interpreter as the interpreter renders the utterance into Norwegian. To the doctor, the content of the next-of-kin’s utterance is only available after the interpreter has interpreted it. As such, the content of the next-of-kin’s utterance becomes disconnected from the ecology – the next-of-kin’s embodied actions and direction of utterance – in which it was produced.

The participants at the hospital know who the patient is. As both the problem and the following interactional trajectory are occasioned by the interpreter’s lack of visual access, the participants do not topicalise the interpreter’s lack of visual access. The doctor simply treats this as an utterance produced on behalf of the next-of-kin, and they proceed in the interaction. Similarly, in the study of mundane interaction, Rintel (2010) found that couples rarely address the technology that occasions the problem, but instead orient to the problem within the content.

Extract (2B) demonstrates how, even when the presence of participants at the ward has been made clear, the participation framework is locally negotiated based on the verbal interaction. Limited visual access to the participants at the ward results in the interpreter becoming more dependent on auditory cues when choosing relevant address terms. The next extract is from a later point in the same interaction as Extract (2A). The meeting is approaching its conclusion and the doctor has just suggested a time for their next appointment.

Again, the doctor directs his utterance to the patient using gaze, while the interpreter selects a pronoun addressing the wife. The doctor asks if the patient has any questions (line 1). Before the interpreter produces a rendition, the next-of-kin begins to produce a question in Vietnamese (line 3). The interpreter has only access to the verbal utterances, not the visual ecology that frames the activity, and makes her linguistic choices on these grounds. From the interpreter’s point of view, the doctor’s gaze is available, but not the person he is gazing at. By the time she begins to produce an interpreted rendition, the next-of-kin’s beginning utterance becomes a part of the interpreter’s source of information for establishing who is the addressee. If the interpreter were to assume that the question was most likely directed to the patient, and thereby selected the male pronoun in this specific context, she would not only repeat the doctor’s utterance, but actively allocate the turn to someone other than the current speaker. In her rendition of the doctor’s question, the interpreter uses the female pronoun (line 4).

In video-mediated interaction, the video image acts both as a link between the participants and as a central tool for the participants’ accomplishment of activities (Mondada, 2007, p. 53). The extracts in this section have demonstrated how the interpreter’s visual access to activities at the ward informs her work. The participants at the ward did not make explicit changes in the participation framework or provide relevant information about the setting. When trouble arose, the interactional problems were dealt with without addressing the participants’ asymmetric visual access or making adjustments to the visual materiality of the setting.

By rendering only parts of the participants and their surroundings visible to co-participants, video-mediated interaction is asymmetric (Arminen et al., 2016; Heath & Luff, 1993). The following examples demonstrate how participants attempt to use embodied resources to organise the interpreted interaction. However, the visual ecology does not afford them the visual access to each other that they presuppose they have. Their embodied actions therefore go unnoticed and they resort to auditory resources to organise the interaction.

In extract (3), the participants have trouble coordinating the interpreter’s turns, and the doctor attempts to use gesture to solve this problem. The doctor and the patient are seated in front of a desktop unit in the hospital ward. The interpreter can only see parts of the doctor, and the patient is seated beyond the range of the camera. The participants at the ward only have visual access to the interpreter’s head. The doctor and the patient are discussing medication. After some problems coordinating turn-taking, the doctor uses gaze and gesture to allocate the turn to the interpreter.

The medical professional is visually oriented to the patient and discussing medication. He asks if it is okay for her to continue “med dette her” (with this) (line 3). After a short silence, the patient produces a minimal response (line 5). The doctor produces a delayed self-repair to specify the referential expression used, “med medisinen” (with the medicine) (line 6), and as such pre-empts misunderstanding. The interpreter’s audible in-breath, a commonly used pre-beginning signal in the data, becomes audible to the participants at the ward during the doctor’s self-repair (line 6–7). The doctor completes his utterance. After some silence, he turns to the screen (line 8). After further ensuing silence, he gestures to the screen with his right arm stretched out and his palm facing up (line 8), seemingly handing something (Streeck, 2009) to the interpreter (fig 3). However, the doctor is only partially visible to the interpreter. The doctor’s arm is not extended within the area captured by the camera, and is therefore not visible to the interpreter (fig 3). The silence following the doctor’s turn and orientation of gaze itself is not enough at this point to elicit interpreting. The doctor and patient begin to laugh, seemingly at the lack of interpreting. The interaction has come to a momentary standstill. The doctor finally retracts his arm and laughingly states that “vi” (we, first person plural), referring to the patient and himself, just have to use “deg” (you, second person singular), using gaze to refer to the screen-interpreter (line 11). The interpreter then begins to interpret.

Although the participants in the interaction have not secured compatible views of each other for the interaction, the doctor uses embodied resources in the organisation of interaction. When the interpreter does not take this up, i.e. they do not begin to interpret, the doctor proceeds to allocate the turn to the interpreter verbally. As such, although the patient has already responded to his utterance, the doctor treats the lack of interpreting as problematic and his embodied action as inefficient. To solve the problem, he explicitly allocates the turn to the interpreter by referring to the interpreter and the interpreting. Explicit referrals to the interpreter do not occur frequently in the data. The interpreter does not respond to the doctor’s utterance in Norwegian or account for the lack of interpreting or missed cue. Rather, she produces a new pre-beginning signal, an audible in-breath, and begins to interpret.

Extract (4) is from the same meeting, from the opening phase. This extract demonstrates how the interpreter uses gesture to initiate the transition to a next activity. However, the participants’ incongruent views of each other result in the gesture going unnoticed. The interpreter has asked in Norwegian if she can provide information about the interpreter’s role, and proceeds to do so in Thai.

As the interpreter’s description of her work comes to an end, the patient responds to the interpreter’s utterance in Thai (line 4). After a longer silence, the interpreter gestures toward the screen displaying the participants at the ward, seemingly signalling the completion of her utterance (fig 4.1). To the participants at the ward, the interpreter’s gestures are not visible, because the interpreter’s hands are below the camera angle (fig 4.1). To the doctor who speaks Norwegian, the vocal production of utterances in Thai and aspects of the turn-taking are available, but the content of the interpreter’s utterance and the patient’s response are not. After some silence, the doctor signals the possible closing of this activity by first asking “okay?” (line 9), before asking if that was okay (line 12). As such, he expresses uncertainty regarding the completion of the previous activity, which was carried out in Thai. The doctor’s lack of access to the interpreter’s gestures has consequences for the unfolding interaction. The doctor solves the problem of his lack of knowledge regarding ongoing activities by means of a verbally produced utterance.

The participants produce embodied actions that presuppose that they have views of each other that are compatible for the efficient uptake of such embodied cues. When the embodied actions do not receive uptake, the participants do not adjust the embodied action according to the visual ecology (as in Licoppe et al., 2017), nor do they address the incongruent views or make adjustments (similar to findings by Rintel, 2010).

The previous extracts have demonstrated how the visual ecology may serve multiple purposes in the organisation of video-interpreted hospital encounters. When participants do not ensure visual access to each other that is appropriate for the activities being undertaken, this may cause problems in the interaction. While the participants solve the interactional problems that occur, they do not topicalise the asymmetric views they have of each other or the visual materiality of the setting. The opening phases of these meetings are possible spaces for making adjustments to the setting and securing an appropriate visual space. However, the professional participants rarely mention this during the opening phase of these meetings.

Extract (5) demonstrates how the doctor (DR) topicalises the interpreter’s (INT) visual access to the ward. However, this does not lead to a collaborative configuring of a visual space for interpreting. As contact is established between the ward and the interpreter’s location, the doctor is only partially visible to the interpreter, and the patient (PAT) is seated beyond the range of the camera. The interpreter’s head and shoulders are visible to the participants at the ward. The interpreter has said some words explaining her work, and the participants are ready to move on to the next topic. The interpreter is holding a pen, indicating that she is ready to interpret.

The participants are about to proceed to a next activity. The interpreter, holding her pen, is seemingly ready to interpret. However, rather than opening the meeting, the doctor moves toward the screen and into the camera frame (line 1) and asks the interpreter if she can see him. This question introduces a new topic into the conversation and thus initiates a topic-proffering sequence (Schegloff, 2007, p. 171). The preferred response to a topic-proffering question is an extended response that topicalises the matter introduced. Prior to the point at which the doctor moves closer to the camera frame, the interpreter has seen only half of the doctor. Furthermore, she has no visual access to the patient. Still, the interpreter answers the question minimally, confirming that she can see the doctor, and looks away from the screen (line 2), thereby rejecting visibility as a topic to be addressed in more detail.

The interpreter did not design her response as a dis-preferred response. However, the doctor’s receipt indicates that it was indeed unexpected, unlikely or even inappropriate. With his gaze still oriented to the screen-interpreter, the doctor begins saying “ja ve-” (okay-) and laughs (line 4). “Ja vel” serves as a third position receipt, indicating slight misalignment with the previous turn, the interpreter’s response. Grinning, the doctor continues, while laughing, “ikke så interessert” (not that interested), expanding on his turn, and then continues to laugh (line 5). Still holding the pen in her right hand, the interpreter claps her hands together, coinciding with the completion of the doctor’s utterance. She then bows her head, laughing visibly more than audibly. The doctor’s laughter following his utterance begins with an equivocal initial sound (line 6), an audible, vocalised in-breath. This format, with an utterance followed by laughter, is found to initiate laughing sequences (Jefferson, 1979). Although the doctor’s remark indicates that the interpreter responded in an unexpected manner, the invitation to joint laughter facilitates a joint display of alignment before the meeting proceeds. The extract demonstrates how, in a case where the doctor topicalises the interpreter’s visual access, this does not lead to a collaborative configuration of a visual space for the interaction.

4. Discussion and conclusion

In this article, I have investigated how participants in video-interpreted medical encounters orient to and use visual ecologies in the organisation of interpreting. The analysis has demonstrated how visual ecologies in video-interpreted hospital encounters serve multiple purposes for participants in the interaction. The visual affordance may enable access to information relevant for the accomplishment of the interpreters’ work. The participants presuppose that the media affords an efficient use of embodied actions. However, the participants do not always ensure that their views of each other and each other’s surroundings are congruent with the activities and actions they are attempting to accomplish. The interaction in these settings is quite complex, as it is not just video-mediated, but also interpreter-mediated. Due to video-mediation, participants’ utterances may be disconnected from the ecology in which the utterances are produced (Luff et al., 2003). Similarly, due to the multilingual nature of the interaction, the linguistic content may become disconnected from the embodied actions that encompass the linguistic content in the original utterance, such as gesture and gaze (see, e.g. Extract (2A)). This can cause complications for participants when making sense of participation frameworks and co-participants’ actions. While the participants’ lack of or incongruent visual access to each other may cause problems in the interaction, they do not attribute the interactional problems to the insufficient visual ecology or make adjustments to the setting. They simply solve the immediate interactional problem and proceed.

Video recordings made by participants in video-mediated interaction, and what they capture and share with remote co-participants, make possible a space of collective action (Mondada, 2007, p. 52). The visual image may be essential for the accomplishment of some professional activities, e.g. during the physical examination phases of telemedicine consultations (Pappas & Seale, 2010) or in surgical settings where laparoscopic surgery, remote expert online advice and remote learning are made possible via the video image (Mondada, 2007). What is a relevant visual ecology for the collaborative accomplishment of interpreting, and how this visual materiality does in fact inform interpreting and the interpreter’s work, might not be entirely clear to the participants in the interaction. While problems hearing might easily be associated with problems in accomplishing interpreting, the participants do not readily connect interactional troubles to insufficient visual access for the accomplishment of ongoing activities.

According to Schutz, the reciprocity of perspectives, in addition to building on the idealisation of the interchangeability of standpoints, builds on the idealisation of the congruency of the system of relevances (Schutz, 1953). While a person’s biographical situation determines at any given moment her purpose at hand, the idealisation of the congruency of the system of relevances suggests that the differences in perspectives originating in people’s unique biographical situations are irrelevant for the purpose at hand (Schutz, 1953). The participants in interaction seem not only to presuppose mutual visibility, but to take for granted that they share an understanding of the purpose at hand. The complexity of the setting, and participants’ various degrees of access to a visual ecology, to linguistic content (and knowledge about the differences of linguistic systems), to background knowledge and even to the purpose at hand at a specific moment in a specific encounter, may in some ways challenge the participants in the accomplishment of that interaction. Engaging in meta-talk about the accomplishment of interpreting in a mediated space could be a possible way to reduce some of the asymmetries in the interaction.

Except for interpreters’ requests to explain their work during the opening phase of the interaction (see extracts (1) and (5)), the professional participants tend not to engage in meta-talk about interpreting or the technology involved. In Extract (5), the medical professional provides an opportunity to talk about the interpreter’s visual access. However, the participants do not engage in the collaborative configuring of a visual space. The way in which the interpreters introduce their work is found in textbooks on interpreting (e.g. Skaaden, 2013) and is practised during interpreting studies. Engaging in meta-talk with a participant in one language poses a risk that the meta-talk may be perceived by speakers of the other language as side-talk, possibly excluding the speakers of the other language (Meeuwesen, Twilt, ten Thije, & Harmsen, 2010). Video-mediated interpreting is still not very common in Norway, and the participants engaging in the interaction have varied experience with accomplishing such meetings. Procedures similar to the introduction of the interpreter’s work have not yet been developed for video-mediated interpreting, although the increased use of technology for the provision of interpreting during the COVID-19 pandemic has occasioned discussions on current practices. Over time, the use of technology has been found to cause an evolution of behaviours, e.g. regarding communicative practices (Dourish et al., 1996). Similar evolutions are likely to occur within the organisation of video-mediated interpreting.

This study has provided insights into the organisation of video-mediated interpreting in hospital encounters and how participants in these settings use and orient to a visual ecology in the organisation of interpreting. As such, the study contributes to the body of knowledge describing various professional activities in mediated environments, and specifically to the understanding of interpreted interaction within a mediated environment. Using multimodal conversation analysis in the investigation of authentic video-interpreted hospital encounters provides insights into the complexity of these interactional settings and the variety of semiotic resources on which the participants draw in their interaction.

Acknowledgements

I am grateful to the reviewers for their comments and suggestions and to Jan Svennevig for his input and advice during work with this article. I am grateful also to Jenny Gudmundsen for useful discussions during work with the article.

This work was partly supported by the Research Council of Norway through its Centres of Excellence funding scheme, project number 223265.