Social Interaction

1. Introduction

What constitutes interactional space in face-to-face encounters is the arrangement of the interlocutors’ bodies in the immediate, material environment, along with their orientation to a joint activity and a mutual focus point (Mondada 2013; cf. Goodwin 2000). Mondada (2009, 2013) describes interactional space as a local and embodied construct that is negotiated at the beginning of encounters, sometimes (re)configured, transformed or repaired during activities and activity shifts, and dissolved at moments of departure. As such, it is a praxeological, practical achievement, comprising verbal, material and embodied resources with which the frames and (pre)conditions of interaction are constantly (re)negotiated (Mondada, 2009, p. 1977). Whereas prior research on workplace meetings has shown that the interactional space for organising activities is sensitive to changes in the immediate material context and participation framework (e.g. Deppermann et al., 2010; Mondada, 2011; Nielsen, 2012), little is known about the procedures through which interactional space is managed and modified in technologized environments where participants are not in each other’s immediate co-presence.

While technologies for organising video-mediated meetings have significantly advanced during the past few decades, the utilisation of embodied resources can still occasionally be problematic in these settings due to asymmetric access to co-participants’ environments and bodily-visual behavior (e.g. Heath & Luff, 1992; Hutchby, 2001, 2014; Norris & Luff, 2013; Rintel, 2013). By affecting the availability of social actions in this way, the creation, upholding and adjusting of co-orientation within a shared interactional space present practical problems (see Luff et al., 2014; O’Hara et al., 2011). Drawing on video-recorded data from a large, international company and taking conversation analysis (CA) as the theoretical and methodological starting point, the present study investigates how participants in an enhanced video-mediated setting (a Cisco Telepresence meeting) orient to troubles in interaction and organise their verbal and embodied conduct to accomplish the recovery of the interactional space. These relate to moments in which an auditory or a visual barrier emerges and is followed by bodily displayed noticings. An important feature of Cisco Telepresence is that it provides a life-sized representations of all the participants, thereby creating an illusion of an extended interactional space and a “blended” overall meeting space (see O’Hara et al., 2011). Noticing is addressed here as action-in-conversation, manifested through a visibly intensified embodied and material practice, such as the production of a particular type of facial expression. The participants accordingly make relevant a feature in the setting that hitherto had not been considered relevant (Schegloff, 2007, p. 219). The embodied noticings under scrutiny are made available through the affordance of video, which has the potential to draw the co-participants’ attention to a noticeable and to fix the trouble.

The analysis focuses on two cases: one involving a difficulty in being heard, the other involving difficulty with visual access to a co-participant’s presentation. It shows that, although they do not receive an immediate uptake, the embodied noticings become accountable actions in that they make visible one’s stance towards the trouble and the fracture in the interactional ecology. In both cases, taking corrective action is crucial, and noticing consequently prefaces the recovery of the interactional space. The recovery is then successively and multimodally achieved, involving first and foremost embodied displays and remedial work that is carried out as a parallel activity - i.e. it does not intervene in the ongoing meeting talk. In addition, due to the nature of the meeting, the participants use technology to manage the interactional space in specific ways, as they attempt to draw the attention of certain participants: those who can repair the problem. The findings contribute to understanding interactional space in video-mediated meetings as a complex construct susceptible to participants’ bodily reorientations and movement in their respective locations, and also highlight the interweaving of actions’ social and sequential environments (cf. Mondada 2011). Furthermore, the paper sheds light on the affordances and multimodal resources in technology-mediated settings and the way they inform the practices of space-making (Oittinen, 2018, 2020; cf. Hutchby 2001; Rintel, 2013; Luff et al., 2014).

2. The role of noticings in reconfiguring interactional space

Interactional space is sensitive to changes in the participation framework and the ongoing activity, and it can be reconfigured and modified by orienting to these changes in the moment-by-moment unfolding of interaction (see Mondada, 2013, p. 261). In addition to openings and closings of encounters, moments prone to renegotiations of interactional space are transitions between activities or episodes. Deppermann et al. (2010) illustrate how the accomplishment of a break-like activity in a formal meeting involves moving from one configuration to another, requiring both verbal contributions and the redistribution of the meeting participants’ bodies. The study primarily shows how the emergent course of action that not relating to the official agenda develops and is collaboratively attended to by the participants’ a “double orientation”: they display orientation to the break-like episode while remaining visibly attuned to the work activities (Deppermann et al., 2010, p. 1707). The present paper expands on these notions by examining how the initial moments of divergent trajectories during meetings can be flagged and made accountable by embodied means, i.e. by a visibly displayed noticing. In CA research, a noticing is seen as an action-in-conversation that singles out a feature in the immediate, physical and sequential environment, such as prior talk, a sound, an item on the screen(s) or an artifact in the room (Schegloff, 2007; see also Helisten, 2019). Registering a noticeable is primarily a result of an individual’s cognitive processing (Schegloff, 2007, p. 87), but a noticing becomes a mutually accessible “interactional event” when it is actualised by displays of a changed epistemic status or a “having just noticed” stance (Helisten, 2019, p. 8; see also Heritage, 1984).

The previous literature describes a noticing through both its verbal and embodied realisation. Embodied noticing involves employing subtle, but visibly perceivable, bodily displays, such as shifts in gaze direction, gestures and facial expressions, which project an attempt to secure co-participants’ attention and the need to attend to other-than-current activity (Kaukomaa et al., 2014; Kääntä, 2014; Keisanen, 2012). Whereas some noticings invoke clear trouble-relevance, and thus make visible the need for repair or other types of remedial work, others do not have similar implications. As such, what is important for understanding the role of noticing is the source/outcome configuration that is contextually construed in the interlocutors’ verbal and visual behaviors (Schegloff, 2007, p. 218). As described by Schegloff (2007), noticings are retrospective, because they make relevant some action or element that was not previously considered relevant, and they are proactive in the sense that they project candidate solutions for next actions. In the present study, the noticings under scrutiny draw attention to technical problems that need to be fixed, but also function as prefaces to the interactional work required to progressively recover the interactional space.

Previous studies have shown the situated character of noticings and how they can become interactionally consequential during instructional activities (Kääntä, 2014; Rauniomaa et al., 2018), in mobile settings (Goodwin & Goodwin, 2012; Keisanen, 2012) and in situations where one is involved in more than one ongoing activity, i.e. multiactivity situations (see e.g. Haddington et al., 2014). In her study on noticing-occasioned interventions in mundane multiparty interactions, Helisten (2019) illustrates that noticings that relate to troubles are often produced by other-than-current-speakers, and that they flag the need to secure either the ongoing activity or another, more urgent, relevant activity. Her findings show that interlocutors have a preference for recipient-designed, non-intrusive ways to initiate these side sequences or new incomings. Helisten (2019) highlights how noticings can become legitimate actions only through co-participants’ subtly displayed verbal and bodily alignments, requiring the suspension of one activity while mobilising another. At the same time, this means orienting to a new configuration of the initially established interactional space (ibid.). Studies on the ways in which noticings can lead to (re)negotiating and modifying the interactional space in institutional interactions are still scarce (see, however, Arminen & Auvinen, 2016; Greiffenhagen & Watson 2009; Oloff, 2018). More research is also needed to understand the unfolding of contemporary multiparty meetings in which participants’ attention can be drawn to various features within the sequential and sociomaterial environments.

3. Managing interactional space in video-mediated environments

An increasing body of research has focused on both the affordances and challenges of video-mediated settings (e.g. Halvorsen, 2016; Hindmarsh & Heath, 1999; Hjulstad, 2016; Rintel, 2013; Olbertz-Siitonen, 2015; Ruhleder & Jordan, 2001a; Ruhleder & Jordan, 2001b; see also Mlynář et al., 2018 for an introduction), but the practices employed to manage and modify the interactional space remain little studied. Although various scholars have found that the sequential organisation of actions is often disrupted by technology-related troubles, such as delays and orientation disparities, their consequences for the interactional ecology or the unfolding of ongoing activities are still not fully understood.

Interactional troubles in traditional video-mediated settings have been described as being due to “fractures” in ecologies – in other words, to the difficulties in producing and interpreting actions coherently both in and between the local and distant environments (Luff et al., 2003; Olbertz-Siitonen, 2015). This has been connected to the use of small screens and thereby to distorted access to common resources. For instance, in their study on videoconferences in a holding company, Ruhleder and Jordan (2001a, p. 119) found that audio and video transmission delays can be perceived differently in the diverse locations, leading to difficulties in noticing troubles and identifying the trouble-source. Consequently, in traditional videoconferencing settings, solving troubles as a part of the work activities has proven to be challenging, owing to a lack of access that enables real-time monitoring of each other’s verbal and embodied behavior.

However, it has been shown that more modern systems, which afford less distorting camera orientation and consequently a wider spectrum of resources for their users, can significantly improve the possibilities for mutual coordination of actions (O’Hara et al., 2011; Luff et al., 2014). Furthermore, these systems would appear to support embodied meaning creation and facilitate the accomplishment of collaborative activities. Luff et al. (2014) illustrate how distant and local participants who can see each other at actual size on screen establish co-orientation and referential activities around material objects in an unproblematic manner. These findings have mostly come from experimental or quasi-naturalistic settings (e.g. Luff et al., 2014), yet they provide valuable insight into discussions of interactional space. The present study, in turn, affords insight into the ways in which vocal and visual actions are coordinated for joint activity in authentic workplace encounters that incorporate enhanced video-mediation.

4. Data and methodology

This study draws on meeting data recorded in one of the offices of a large international company in Central Europe in 2012 and 2013. The analysis is based on a video-recorded steering committee meeting, lasting approximately 80 minutes, in which Cisco Telepresence was used to enable collaborative work between the distributed parties. In the company, this particular system is used sparingly and mostly for meetings that are considered of high importance. The important features of Cisco Telepresence are the design and room architecture, which are identical in the ‘Telepresence’ meeting rooms in each location. Each has a short, semicircular meeting table with six seats facing three large screens at eye level, and an additional screen above the three, on which the agenda and other materials can be displayed. Each table has three microphones with a mute function that the local participants can control. The on-screen participants are represented at life size and in a geometrically accurate form, which emulates co-presence (see Arminen et al., 2016). These arrangements create an illusion of an extended space that ranges from the local to the distant environments (see O’Hara et al., 2011).

The present meeting comprises seven participants. One is physically in the recording room, while the others are in two different geographical locations. The recording took place in one physical location, which means that the analysis of the orientations of embodied displays, such as gaze and gestures, depicts a single perspective, in this case that of the local participant, Marja. From her point of view, the others are distant participants whose images are portrayed on screens 1 and 2. The display on one of the screens (“screen 1” in the transcripts) alternates automatically between the images of two pairs of distant participants: Dietmar and Rob, and Olek and Leena, depending on who is talking. Olek and Leena are in the same room with Jaap and Noach whose images can be seen on screen 2 throughout the meeting. Screen 3, in turn, shows the meeting materials, e.g. the agenda and presentations. The others have a screen in a similar position in their respective locations, which is the reason for not distinguishing between Marja’s screen 3 and those of the others in the analyses. The following images illustrate how the participants are seated and presented in Marja’s local environment (Fig. 1 & 2).

5. Recovering interactional space after noticings

The analysis shows how the interactional space is jointly and progressively recovered after noticings that manifest 1) a sudden shift in participants’ orientation to a noticeable, and 2) attention to a potentially problematic feature in the setting. These noticings occur in moments when the sequentiality of conduct has become incoherent because of an auditory or a visual barrier, i.e. when there is a fracture in the interactional ecology that has led to compromising one or several parties’ orientation to a shared focus point in the main activity. The recovery of the interactional space entails dissolving and, along with remedial actions, reestablishing the initial configuration. The two instances analysed below relate to trouble in being heard (Section 5.1) and in seeing the co-participant’s presentation (Section 5.2). In both cases, embodied noticing, as displayed in shifts in gaze direction and body movement, functions as a significant first step towards mobilising attention to the trouble source – a problem that can be fixed by manipulating an object or a device in only one of the meeting rooms. The analysis illustrates how the participants remain attentive to the specificities of the video-mediated interactional space that are relevant to both the course of interaction and the emergent ongoing activity.

This section analyses a sequence in which two distant participants, Dietmar and Rob, cannot be heard by the others because of a muted audio-connection. This becomes visible when Dietmar starts to speak at the same time as Marja, who is not in the same room. The recovery of the interactional space is prefaced by Rob’s noticing of the trouble and managed as a separate, parallel activity that takes place simultaneously with the on-going meeting talk.

In the first extract, the topic of shipping is being discussed. Since this is Noach’s area of responsibility, he has prepared materials for his presentation, which are displayed on screen 3. Three minutes prior to the extract, Dietmar has clicked on the table microphone and muted it in the meeting room where he and Rob are physically present. This caused the small lamp on the side of the table microphone to change color from green to red. Ten seconds before the beginning of the extract, Dietmar has shifted from a leaning body position to an upright one, thereby showing preparedness to take part in the discussion. In the extract, Dietmar fails to take the floor verbally (lines 5–9), which prompts Rob to find the trouble source and unmute the microphone. Embodied displays, such as gaze and shifts in body position, thus become important resources with which the recovery of the interactional space is initiated and made intelligible to Dietmar.

The second subsection analyses a sequence later in the meeting, in which Dietmar is expected to share materials on another topic, but the others are unable to see his PowerPoint presentation. At first, the problem goes unnoticed by the current speaker and verbally unannounced by the others (cf. Schegloff, 2007, p. 87). In this case, however, several participants in the different locations make use of visibly intensified embodied practices to indicate their noticing of the trouble. These consecutively produced noticings, especially those by Marja, Noach and Jaap, lead to a dissolving of the shared interactional space and the participants remaining in a kind of liminal state for a short period. In other words, they enter into a dispersed configuration in which several participants orient to individual business in their local space. Due to the length of the event, the analysis is presented in two parts: sequences that show 1) how the problem of not seeing the presentation first emerges and goes unnoticed by Dietmar (Extract 2); and 2) how, shortly thereafter, the problem is attended to collaboratively (Extract 3). What is noteworthy in the extracts is that, instead of initiating repair in slots in which it would be sequentially suitable to do so, such as pauses (see e.g. Helisten, 2019, p. 6), the other participants allow Dietmar to do the noticing himself (Schegloff, 2007, p. 87).

The next extract begins after Marja has suggested moving on to the next agenda item, a problematic issue concerning the company’s delivery processes that needs to be resolved by the members of the steering committee. At this point, the screens in each location continue to show Noach’s presentation. To change this, Noach must unplug his laptop cable and Dietmar must enable sharing mode via the Cisco Touch Panel, which controls the screen view and is on the table behind his laptop. Immediately before this extract begins, Olek, who is physically in the same room as Noach and Jaap, makes these preparatory actions explicit and asks Noach to pull out the cable. The extract begins when Dietmar acknowledges this verbally (lines 1–2), and Noach, who has been writing, similarly orients to the activity shift and begins to remove the cable. However, Dietmar does not press the touch panel to enable access to his presentation, which the subsequent noticings of Marja, Noach and Jaap make relevant (lines 7–9). The extract shows that, as a result of Dietmar’s inability to recognise the problem or his co-participants’ noticings, the initial interactional space is reconfigured, but not recovered.

6. Concluding discussion

This study has investigated noticing-prefaced recoveries of the interactional space in a video-mediated business meeting. Noticing has been treated as action-in-conversation that singles out a feature in the physical and sequential environment that has not, up to that point in the interaction, been considered relevant (Schegloff, 2007, p. 219). The article has focused on particular moments when conduct has become sequentially incoherent, i.e. when there has been a fracture in the interactional ecology, owing to either an auditory or a visual barrier, and one or several parties’ orientation to a shared focus point has thereby been compromised. The two instances analysed highlight the recovery of interactional space as an emergent and collective accomplishment in which embodied noticing plays an important role. The analysis shows that while noticings are made available through video by intensified bodily-visual practices, they are not immediately attended to in the moment. This manifests the complexities of the video-mediated setting and its potential consequences for the organisation of (inter)action.

In the present Cisco Telepresence, the participants draw on a variety of verbal and bodily-visual resources to coordinate their actions, both sequentially and spatially. The analysis of the first instance (Subsection 5.1) illustrates how trouble in being heard compromises the participation of one party and is then recognised and solved in a non-intrusive way by one participant. The recovery of the interactional space involves embodied practices, including locating the trouble-source via gaze and remedying the situation by corrective body movement and making technological adjustments, i.e. unmuting the microphone (cf. Arminen & Auvinen, 2016). In the second case (Subsection 5.2), trouble is experienced in the form of the co-participant’s presentation being visible. The other-than-current speakers use gaze and facial expressions to display their having-noticed stance, after which they accomplish a new liminal-state configuration of the shared interactional space. Although the current speaker seems to perceive the long silences as “trouble flags” (Ruhleder & Jordan, 2001a), he does not at first notice the trouble source. Both instances show how, although disregarded in the moment by the current speaker, trouble-relevant noticings occasion a shift in co-orientation, which has consequences for the way the interactional space is renegotiated. In addition, in neither case is noticing followed by a verbal initiation of repair, indicating the participants’ preference for implicit ways of making someone accountable for technology-related trouble. Here, unlike in mundane face-to-face interactions (see Helisten, 2019), noticings do not lead to a suspending of the main activity; instead, the emergent remedial work is managed as a parallel activity along with the meeting talk. It may be argued that this indicates some level of preference for maintaining progressivity over (re)establishing intersubjectivity. However, more evidence is needed to substantiate this view.

This study has shown that space-making is a multilayered and constantly evolving achievement that involves not only multiple spatialities but also the ability to make sense of co-participants’ orientations to them on a moment-by-moment basis (see Mondada, 2011). The enhanced, video-mediated environment of Cisco Telepresence, in which everyone can see each other in life-sized representations, seems on the one hand to facilitate meaning-making via embodied conduct, and on the other hand, to present challenges for the production and interpretation of actions. This is due to the complexity of the situation, namely having to monitor the sequential positioning of relevant verbal and embodied actions in various locations, as displayed on several screens in the different meeting rooms. Overall, in this respect, the participants manage their orientation to each other and the agenda screen (screen 3) smoothly. However, as shown in Extracts 2 and 3, treating subtle noticings as relevant in the interaction and identifying the exact moment when the need to recover interactional space occurs can be difficult. This requires skillful use of the technology in question, but also understanding of the ways in which one’s own actions shape and are shaped by the sequential, material and technological environment (see e.g. Hjulstad, 2016; Hutchby, 2001, 2014). In addition, to solve the troubles relating to auditory and visual barriers, one must have access to controls for the sound or the visuals, and recognise that there is a problem that needs to be fixed.

This study presents initial insights into the process of recovering the interactional space in video-mediated interaction by focusing on a specific organisational context. There is a need for more research drawing on larger data sets that illustrate the practices of space-making in different settings, and for further understanding of coordinated actions as key to the smooth running of technology-mediated meetings. The findings can yet be applied by practitioners and IT designers aiming at improving workplace communications across physical distances.

Acknowledgements

I wish to thank the two anonymous reviewers and the special issue editors, as well as Pentti Haddington and Leila Kääntä, for their constructive and insightful comments on the earlier versions of this manuscript.

Appendix