Social Interaction

2.1 Establishment of the EMCA research project

One of our study sites is part of an interdisciplinary research project conducted at a department of gastroenterology in a large public university hospital in the greater Tokyo area, Japan. This project was launched with the aim of studying human action conducted within a highly technical and technologically skillful environment. The fieldwork period ran from 2014 to 2016 and recorded 15 cases of gastroenterological surgery, generating a data corpus of 50 hours of video. The operations that were video recorded lasted a minimum of 2 hours, with some cases lasting more than 5 hours.

The data collection at the site was launched after obtaining ethical approval where the researcher was invited to record operations in which a novel 3D surgical navigation system, a 3D image reconstruction of the patient’s internal body organ structures based on scanned images, was used to help the surgeons grasp and predict where the tumors are located in relation to the neighboring blood vessels (Oshiro et al., 2017). Because of the unique technological features of the surgery, the practitioners’ interactions sometimes involved a discussion of using these navigation systems, and the researcher was expected to learn about and gain procedural knowledge of how these technologies work in addition to more general understanding of surgical terminologies. Typical operations involved the partial or entire removal of organs such as livers, pancreases, or gall bladders. Therefore, decision making as to where and how to perform excisions (which is primarily based on the sensorial feedback and information one would receive from the operation) became an omnirelevant problem for the surgeons.

Our second example concerns work performed in what is categorized as a hybrid operating theater in a Swedish hospital. This fairly recent development in operating suites combines minimally invasive surgery (often relying on X-ray guidance) with open surgery. In these setting, surgical team members with differing expertise and different working methods can collaborate in new constellations.

Due to this novelty, the operating suite used for the study, built as a learning facility, was equipped with multiple ceiling-mounted video cameras and the capability to stream all output from computers and imaging equipment to a monitoring room located elsewhere in the hospital. When this facility was first constructed, the staff raised concerns about the possibility of having their work remotely monitored; therefore, a number of procedures were established to create a feeling of safety and control. As a first measure, recording can only ever be initiated from the control room adjacent to the operating room; it cannot be turned on remotely. Additionally, a number of “On Air” signs are mounted within and outside the operating theater, clearly indicating when recording is ongoing.

Another important aspect of this operating room was its ventilation system and how it restricted recording. Additional bodies and equipment present within the surgical suite would reduce the effectiveness of the ventilation. There could, therefore, be hygienic consequences when recording inside the operating room, even with sterilized equipment kept at a distance. As a non-medical specialist, it is difficult to assess how great a risk this presents. Given the option of making remote recordings, this path was chosen for the project.

2.2 Role as a researcher in the fieldwork

In addition to the details of how the researchers undertook the fieldwork at their respective sites, creating and maintaining rapport with the participating surgeons was equally important to conducting research in such highly specialized settings.

In the Japanese case, the project was initiated by surgeons interested in developing state-of-the-art 3D imaging technology for navigation during an operation. The researcher was later invited to participate in the group due to her specialty in conversation analysis. Even though the initiative came from another party, it was important to demonstrate how research using conversation analysis can be conducted effectively and prove valuable in terms of describing ‘the seen but unnoticed’ (Garfinkel, 1967, p. 36) aspects of a surgical operation. Thus, the researcher presented her findings at several medical conferences and joined research meetings with surgeons whenever possible. Because of this position, the researcher was able to consult with surgeons and technicians on the data and technical aspects of the surgical procedures.

As the sole field researcher, the first author managed all the recordings in the Japanese setting. However, if a scheduled surgery could not be attended by the researcher, staff members of the surgical department helped with the recordings. Because the operating room was not large enough for the surgical team as well as an observer, participant observation was not an option. Therefore, during the surgery, the operating room was only visited by the researcher to check on the equipment. This arrangement was possible only because of the researcher’s enduring efforts to create and maintain rapport with all concerned parties.

On the other hand, the Swedish project was set up as a collaboration among an active surgeon, medical physicists responsible for radiation safety at the hospital, and the researchers. By jointly formulating the project’s goals, much of the required legitimacy for conducting the research was formed at this initial stage. In this case, however, additional communication with the hospital’s management team, union representatives, and the staff at large (i.e., the team of anesthesiologists, nurses, and vascular surgeons) was required to establish an understanding of the project’s goals. The requirement to gather informed consent from all parties potentially affected by the study entailed a four-month long process of meetings before any recordings could be initiated.

Hospitals are highly sensitive areas with specific ethical, legal, technical, and hygienic standards or requirements, and a researcher must be trusted not to interfere with medical practices so as to not jeopardize their integrity. Establishing a level of trust in the researcher’s ability to professionally conduct the research is thus paramount. This can be a long process involving constant demonstrations of how one is capable of providing relevant feedback or observations, which reflexively leads to the building of rapport with practitioners.

2.3. Overcoming technical challenges for making video recordings and analyzing surgical interaction data

As the setup for medical surgery usually involves advanced and cutting-edge technologies, conducting fieldwork for EMCA studies imposes significant challenges on the researchers. In addition to adapting to highly skilled settings, the researchers also have to meet the requirements of sterilization and sensitivity to the type of work that they are studying. In this section, we describe how the researchers in each setting dealt with difficulties indigenous to the type of work they were studying.

At the Japanese site, video recordings were typically conducted using two cameras. The cameras were placed diagonally across the room to capture all participants’ fronts (see Figure 1). We set up one camera in a corner and the other on some equipment installed on the opposing wall to minimize any obstruction of the medical team’s movements and to maintain the environment’s sterility. Even though all participants were technically recorded from the front, they often gathered closely together during the operation. The result was that some bodies would occlude those standing on the opposing side of the operating table.

Figure 1.

A central part of the operating room was the surgical monitors displaying the close-up shot of the operation field captured by the operation camera. The monitors hung from the ceiling above the patient’s head. The same footage was available to the researcher after the operation; however, this material did not have any sound. Working from this source made it strenuous to coordinate the images with the conversational data (though it was not impossible). The surgeons declined wearing lapel microphones due to sterility concerns and the possibility that the extra equipment might restrict their movements during operations. For these reasons, wireless microphones were instead mounted on the surgical camera in the operating area, so that the external cameras could record the talk.

Under these conditions, a number of difficulties arose during the subsequent analysis. The noise of the operating room was disruptively loud, with sounds emanating from the various activities, equipment, and people moving about the operating room, and the surgeons’ masks made their conversations difficult to hear. Additionally, on occasion, the equipment, such as the surgical lights and the various types of monitors, interfered with long shots of the scene, making it difficult to see what the surgeons were doing. As a consequence, the researcher had to make the best of the suboptimal recordings. In hindsight, an IC recorder could have compensated for the lack of clear sound. However, at the time, the primary consideration was not to interfere with the surgical field, so no additional recording equipment was introduced.

In the Swedish context, the cameras mounted on the ceiling were of very high quality and could be remotely controlled with respect to pan, tilt, and zoom. A number of XLR connectors for microphones were also spread across the ceiling and on pendants, enabling the selective advance placement of microphones to focus on specific team members. Even so, poor sound quality remained an issue, similar to the Japanese setting. However, with the help of the technicians who had built the sound and video installations, we were able to optimize the sound and noise filtering during the ongoing surgery.

Nevertheless, the sound conditions, even for the participants in the setting, were challenging. The large space combined with the sounds of fans, a large robotic arm carrying an X-ray sensor, and additional noise-generating equipment created a soundscape of varying complexity. At certain times during the recordings, all the voices could be heard clearly, but at other times, particularly when several different groups of staff were talking at the same time, it was difficult to differentiate the separate streams. To counteract this, it would have been helpful to use individual lapel microphones for staff members of interest. However, this would have required wireless microphones. As the operating theater is a technically sensitive setting with respect to the risk of interfering with life-sustaining equipment, as in the Japanese context, we did not explore wireless technology options. Again, we chose to use the existing technical solution despite its potential drawbacks.

For remote viewing unique to the Swedish case, a conference room was fitted with two large monitors and a control panel. Once a session had been initiated from inside the operating room, it was possible to select from among twenty different feeds (e.g., camera, computer, X-ray, ultrasound, and endoscope) and project one feed on each monitor. A major benefit of this setup was that all sound coming from the operating room was automatically integrated with whichever video stream was selected. This had tremendous beneficial consequences for the later stages of synchronizing the sources.

The majority of the surgical work was being carried out in front of a computer monitor hung above the patient, with one surgeon controlling the inserted instruments by hand. To capture as much of the central area of the surgical work as possible, it was decided to use one overhead camera shot in combination with a video stream from the surgeon’s main working monitor. The information displayed on this monitor was found to be essential for following the events and to make sense of the progression of the procedure. These two recordings were then combined into a single timeline using Final Cut Pro editing software. By maintaining the original size of the surgical monitor feed, the same visual access the participants had was available for the secondary analysis.

Another factor when making these recordings was the duration of the surgical procedures. The shortest recordings were two hours long, while the longest recording was over 10 hours long. For the research project, 12 procedures were documented, totaling 58 hours of recording. As a consequence of having very long recordings, Final Cut Pro was found to be the best software for continuing the analysis. Unlike most video playback software, video editing suites typically come with scalable timelines. When this timeline is zoomed out, one can quickly look for events and changes that occur over many hours, and when it is zoomed in, one can identify differences down to individual frames. The added feature of creating markers and other forms of annotation also makes it possible to build a preliminary structure for the analysis into the video materials. In this way, for instance, every occurrence of artificially induced apnea (115 cases) was identified and annotated, and later analyzed and reported (Ivarsson & Åberg, 2020).

Even with this kind of high-quality visual access and the technologically supplemented help, the surgical work and its analysis were not without complications. The selected examples analyzed below in 3.1 illustrate some of the concerns regarding the different sensorial aspects of the studied practices. As we aim to demonstrate, these are concerns, for members of surgical teams as well as for analysts, regarding the accountability of the members’ conduct as the focus of the study—just as in any other perspicuous setting.

3. Observing and accounting for multisensoriality in surgical operations

3.1. Difficulties and challenges in analyzing the multisensorial and multimodal interactions of technical surgical operations

As mentioned, a surgical operation is a multisensorial field. Surgeons dissect, open, and ligate blood vessels and organs. Most of this activity requires the surgeons to “see” various objects and make judgments based on their professional vision (Goodwin, 1996, 1997; Kuroshima, 2018). However, the work also relies on other human sensations such as touch, hearing, and even smell in certain circumstances. For example, when dissecting tissue, tactile feedback from the object is an important resource for a party to assess the possibility of continuing a dissection (see Excerpt 2).

Typically, researchers face serious challenges when analyzing such perception-based actions (Nishizaka, forthcoming). Akin to touch in a professional activity, the subjective experiences mediated via one’s other sense organs are not directly available to other parties in a non-interactional and technical sense. However, as the activity at large cannot be accomplished by just one person’s conduct (Schegloff, 1968), multisensorial aspects of the work have to be brought to the collaboration by the participants themselves. In this way, private perceptions are rendered observable for others as well. Subjective experiences are made available for observation, assessment, and comments to achieve a shared understanding of the surgery at any moment in time.

By making sensations available and analyzable for co-present participants, the subjective experience via one’s sensations is also made available to the researcher observing the activity through a camera lens. For example, professional massage therapists’ tactile perceptions are used as resources to produce not only diagnosis but also the social actions of acknowledging and confirming the problems mentioned by patients (Kuroshima, 2020). Although we have better access to the participants’ visual practice, as we can arguably “see” what they are doing by looking at objects (cf. Gibson, 1966), other senses are not necessarily unavailable as long as they are made public through concerted actions (i.e., the rational accountability of practical activities) (Garfinkel & Sacks, 1970; Nishizaka, 2017; see Streeck’s [2013] discussion about the living body). This point is demonstrated in the following analyses. That is, each analysis shows how the skillful and professional organization of the multisensorial work of surgical operations can be approached. The subsequent analyses, the first three from the Japanese setting and the remaining three from the Swedish setting, are dedicated to describing and analyzing how various types of multisensorial action constitution (e.g., visual, tactile, and auditory perceptions) manifested in interaction. In these praxeological accounts, the common-sense rational properties of multisensorial conduct will be made available. As will be demonstrated, even such distinctly technical activities as surgical operations are made observable through the accountability of practical actions (Garfinkel, 1967).

3.2. Making a judgment based on visual perception

A practical problem for participants in a multi-party activity setting is the establishment of intersubjective ground. One way to deal with this problem is by displaying one’s perceptions in talk. Excerpt 1 illustrates this point. In this segment, the surgical team seems to have reached a critical point for their operation (the partial surgical removal of the liver). At the start of Excerpt 1, the main surgeon (D1) and the two assisting surgeons (D2 and D3) are looking at the operating area. D1 starts making a judgment call to the group (line 2) and subsequently provides an account for doing so (line 4). With his deontic authority of the matter in question—in his position as lead surgeon, he is expected to have the right to propose the next course of action—this judgment is understood as a proposal for the team to prepare to stop (Stevanovic & Peräkylä, 2012). Although the researcher’s camera only captures the surgeons from a distance and their upper bodies (see Figure 2), the surgical area becomes available through the surgical monitors (see Section 2.4 for the arrangement of recording). Though the image of the surgical monitors cannot be quoted in the paper for ethical reason, this view aided the researcher in analyzing the members’ methods of accounting for their visual perceptions as the surgery proceeds.

Excerpt 1.

In line 2, D1’s proposal to stop presents his judgment via his visual perception using deictic terms and ending the turn with a quotation marker -tte, which is used for making a report (Hayashi, 1997). In this way, as akin to Goodwin’s (1996) professional vision, he presents his perspective on the current status of the operation. Despite his deontic authority, he further accounts to justify how his judgment is reasonable by explicating how he sees the blood vessels structure at this point (lines 4 and 5). In particular, he couples his bodily display of how he sees the current status of the operation with the particular setup of the operation, that is, by showing his gaze direction towards the surgical field and pointing at the blood vessel with his scalpel (Figure 2) (Goodwin, 2013, 2018). By producing an action based on his visual perception in this way, D1 invites them to assess the same object, which would establish an intersubjective ground for the current activity (Kuroshima, 2017). The proposal was accounted for most likely not due to the reduced deontic authority, but due to the nature of the proposal which is constructed via a speaker’s perception. Indeed, in line 6, D3 endorses D1’s judgment by repetitional agreement via visibly showing that he is looking at the surgical field; thereby he also constructs his agreement based on his own visual access to the object in question (cf. Nishizaka, forthcoming). After this endorsement is obtained, D1 proceeds to make more specific proposal to discontinue the procedure than the one in line 2 (line 8). During this proposal, D1 shifts his gaze from the operating area to D2 to build a new course of action, which is accepted by D2 in line 10.

In this example, the surgeons’ professional vision becomes available not only to the recipients of these actions but also to laypersons, such as the researcher, through an examination of the accountability of the participants’ practical actions as they are rendered visually accessible to the recipients and sequentially positioned with one another.

3.3. Complaining based on tactile perception

The following excerpt was recorded during a cholecystectomy (i.e., the surgical removal of the gall bladder). D2 is dissecting part of a gall bladder and D1 is watching the work on the monitor showing the laparoscopic image. D2 has been trying to dissect tissue with an electronic scalpel for a while. As a continuing state of incipient talk (Schegloff & Sacks, 1973), D1 remarks that the scene he is observing is noteworthy because of a unique feature of the patient’s organ and seeks agreement from the free participant, D3, by shifting his gaze from the monitor to D3 (line 1). As the prompt to comment on the scene is made, D2, still intensely looking at the monitor (Figure 3), complains about the tissue being especially katai (hard) and that he cannot proceed smoothly.

Excerpt 2.

As a response, D1 agrees with D2 and further validates his complaint by offering an account of why the tissue is hard (line 4) (i.e., because the patient’s organ is inflamed); thus, treating the complaining as reasonable to be initiated at this moment. Although the tactile feedback of the tissue (which is hard) is only received by D2, in this case, D1 is clearly able to endorse D2 by observing how D2 visually presents his problem with the procedure through his description of his tactile perception, body, and manipulation of tools, thus making the scene accountable via a complaining action. This exchange is a clear illustration of how one’s sensorial reception becomes accessible to others based on the formulation of one’s action (i.e., a complaint).

3.6. The practical problem of understanding touch and tension

One of the things missing for us as onlooking analysts is the tactile sensation of the instruments as they are being manipulated. There are obvious differences in stiffness of the existing varieties of wires and catheters. Furthermore, the insides of arteries can be sensed and probed in various ways. From discussions with the vascular surgeons, we have learned that this tactile modality is central to their profession. Handling the instruments inattentively can lead to accidental punctures in fragile vessels and potentially fatal complications. It seems, however, that the added sense of “presence” or understanding achieved by this remote touching does not go beyond the person actively involved. This means that even a skilled surgeon watching from the sidelines must resort to visual inspection.

A further complication for the analysis of this sensory modality is that the tactile performance involved in this task is an exploratory mode of sensing, aligning with what Katz (1925, 1989) described as active touch. This means that the hands and fingers, as sense organs, must be actively involved in the touching for the kinesthetic sensation to appear. As the movements cease, what has been touched is lost.

An example relating to this problem is illustrated in excerpt 4, where surgeon 1 (SU1), the main surgeon, is operating the guidewire by way of rotation and pulling and pushing it through the sheath. The resulting actions are mainly visible on the screen facing the surgical team (Figure 5). In line 3, the surgeon initiates a form of complaint leading up to a formulation of his actions: He is trying to make contact with the outer parts of the stent graft, to “feel” where it is. He begins, however, by presenting the problem in terms of visual distance, thereby making it accessible to the other participants. Before she is interrupted, the resident is also on the way to suggesting a solution (line 4).

Excerpt 4.

  Open in a separate window

Figure 5.

The contact that SU1 is seeking can be potentially observed on the visual display; however, without tactile feedback, the imagery remains ambiguous. Thus, there seems to be a dividing line between the person involved in the active touching and everyone else. Assisting surgeons and video analysts alike are effectively barred from a full understanding unless they take control of the instruments. To counteract some of the risks in relying on “private” experiences, the practice has developed additional methods to make the actions accountable. This is illustrated in 3.7 and 3.8.

3.7. Reconstructing three dimensions

In excerpt 5, surgeon 1 is searching for the opening of the stent graft lumen by manipulating the instruments. As described in section 3.5, those actions are visible on the monitor. One complication connected with integrating the visual view with the physical reality of the instruments’ location is that the three dimensions of the space are represented by a two-dimensional image. The perception of depth is thus missing. On the X-ray image, depth is collapsed onto a single plane. This means that what appears to be the successful placement of the wire inside the lumen may turn out to be unsuccessful once the X-ray sensor is rotated.

Excerpt 5.

  Open in a separate window

Figure 6.

With SU1 continuously trying to find the correct place for the wire tip (line 1), surgeon 2 (SU2) steps in and begins to reposition the robotic arm carrying the X-ray system (Figure 6). By rotating the robotic C-arm, the resulting view on the monitor is correspondingly shifted in real time. During the rotation, which takes a little over 11 seconds, an added sense of three dimensionality is given to the visual view. The spatial relations between the wire and the stent graft are now somewhat easier to appreciate. Once the rotation is brought to a halt, after a few seconds, SU2 offers his assessment of the work (line 3): “We are extremely close”. This assessment simultaneously acknowledges the ongoing “failure” to position the wire correctly and the progress that has been made thus far. SU1 temporarily halts his attempt to find the right position, which is punctuated with a brief exclamation of frustration (line 4), as though he is dissatisfied with his performance.

3.8. The public display of successful placement

Apart from rotating the X-ray sensor, there are additional cues that can indicate a successful placement. If the wire has entered the lumen correctly, it will always retain a connection with the lip of the opening. As the wire twists and bends its way through the body, this opening provides an outer limit to the wire’s displacement at this particular point. If such a connection can be observed, it is a good indication that the task has been carried out successfully.

In the final excerpt, the operating surgeon is placing the wire in what looks to be a good position. He remains silent, but the resident watching from behind offers a positive assessment (line 1). The assessment is stretched out with a rising intonation, and as it reaches its peak, the first visual indication of contact between the wire and the lumen can be observed on the screen (Figure 7). The resident’s somewhat anticipatory assessment thus tightly coincides with the visual evidence of the event that it remarks. The radiology nurse also provides an affirmative comment (line 2). The resident offers another comment, but it is difficult to hear (line 3). However, as the surgeon slightly repositions the wire, there is again clear evidence of continued contact between the wire and the lumen (figure 8). Without any comment, the surgeon halts his work and leaves the instruments in place. As he stops the flow of radiation by stepping off the pedal, the screen freezes, showing the last image. The scrub nurse now offers the deictic “there” (line 4) and makes a gesture by lifting the palm of her hand twice, thereby mimicking the wire’s contact point with the upper leftmost part of the lumen.

Excerpt 6.

  Open in a separate window

Figure 7.

Figure 8.

In this excerpt, the surgeon is finally finishing the task of correctly threading the guidewire through the lumen, something he has been attempting to do for the past eight minutes. Upon its success, however, he makes no verbal remark but can be seen backing away from the instruments. Instead, it is the other participants who collectively provide the public understanding of the new status of the procedure. As they will now proceed with the next phase of the surgery, this may imply different responsibilities; the marking of this transition is therefore significant.

As illustrated by this case, tactile evidence of physical contact is, in one sense, a private phenomenon experienced only by the leading surgeon. Nevertheless, the surgical practice is simultaneously organized so as to minimize its reliance on private sensations. Such phenomena are systematically made available to the other team members, thus minimizing the risk of errors of judgment and ensuring that the work can proceed in an orderly fashion. Across the range of examples given, we wish to highlight how external analysts can exploit this feature of the organization. As the other team members are given access to the surgeon’s multisensorial analysis, we are likewise granted access to the same level.