Social Interaction

1. Introduction

In the course of their interactions with others and the socio-material world, individuals perform multimodal actions for which they are held responsible. The notion of agency denotes this capacity to act on one's environment, objects, and others, as well as an individual's perception of this faculty (Butler, 2002). Agency can be comprehended as a cognitive and intuitive guarantee that an agent is in control of their actions, avoiding any confusion between the self and the others (Nadel & Decety, 2006). Debates about the nature, conditions, and scope of agency have occurred in human sciences since the Enlightenment and can be traced back to Locke, Hume, and Aristotle. If the level of awareness, reflexivity, and intentionality involved in action projection and design is the locus of vast debate, agency has primarily been ontologically understood as a human trait. In Western philosophy and psychology, objects perceived as devoid of reason, desire, and belief have been left out of the agency dilemma as it was assumed that only persons reflect on and care about their motivations (Frankfurt, 1971).

However, away from these dual (humans are agents, objects are not) or hierarchical (humans have more agency than non-human agents) comprehensions (Frankfurt, 1971), another approach to agency was proposed under the actor-network theory. Its founder, Bruno Latour, challenged the distinction between human and non-human agency. Rejecting both anthropocentric and socio-centric views, Latour defined an actor as "any thing that does modify a state of affairs by making a difference" (Latour 2005, p. 71).

Furthermore, anthropological work on agency has increasingly emphasised the decentralisation of the intentional human subject in favour of a multiplicity of agents, whether human or non-human (De Fornel, 2013). Nonetheless, drawing on the observation of ritual practices of indigenous groups and aiming to dismantle the objectivist Western perspective, anthropological scholars have highlighted two main orientations; while both of these reject the denial of object agency, they diverge regarding its enactment. The first orientation takes an ontological orientation and considers agency as stable. Conversely, the second defines agency as unstable and contextual: animals, plants, and artifacts are only conceived as persons within situated activities (ritual, relational, etc.). For these entities to become agents, a work of perception and categorization must be carried out by human beings within daily or ritual activities (De Fornel, 2013). The latter orientation, with which we align, reveals that agency is a situated, dynamic, collaborative, and temporally unfolding enactment within socio-material interactions (Ibnelkaïd, 2019).

This understanding of agency as a local transient accomplishment is crucial in the digital era, in which the ubiquitous presence of digital artifacts affects individuals' everyday practices. In projecting and designing their (inter)actions, individuals "can treat a technology as a communication partner, as an active part (and actant) of an activity or as an opponent of action" (Krummheuer, 2015, p. 179). Therefore, the affordances of technological devices, in both their materiality and digitality, are not inherent but result from the different ways interactants orient to the artifacts moment-to-moment (Krummheuer, 2015). Thus, it appears more relevant to observe the enactment of situated agency in artifacted social interactions to overcome the ontological dilemma of conceptual agency.

Although the role of material objects in interaction has been considered in early ethnomethodology and conversation analysis (e.g., Garfinkel et al., 1981; Suchman, 1987; Goodwin, 1994), over the past two decades, object use has become a central focus of many studies in this field (Hindmarsh & Heath, 2000; Nevile et al., 2014: Day & Wagner, 2019). Most of these studies have focused on how participants use objects to achieve specific actions or facilitate the progressivity of ongoing activities. This approach to objects-in-interaction is consistent with the epistemological position of EM/CA, which focuses its analysis on the member's perspective. From this perspective, objects cannot be granted agency nor can they be considered members; they can only be analysed as resources or accomplishments. However, approaches that only consider artifacts as objects potentially used by participants to facilitate progressivity do not consider situations in which objects interrupt specific action trajectories or interaction in general.

This ability to disrupt projected actions is especially relevant for technological objects, which may act against users' expectations. For example, a smartphone showing sequence in face-to-face interaction may be suspended or even abandoned if the smartphone 'refuses' to work as expected. In this situation, a technological object's potential agency could be questioned as it affects participants' action projection and (inter)action progressivity. Participants in the interaction may then perceive digital devices as recalcitrant agents.

Therefore, to further explore the concept of situated agency and better understand its local enactment, video-based research on artifacted social interactions must address theoretical and empirical questions such as:

• How is situated agency accomplished and negotiated within the projection and design of joint actions?
• What are the procedures used by participants in attributing agency to artifacts?
• How is situated agency distributed in the interrelations between humans and artifacts in collaborative actions?
• How do digital artifacts, as potential situated agents, enable or constrain embodied actions and participation?
• How are action perception and recognition affected by situated agency?

This special issue brings together video-based studies on the local accomplishment of (human and non-human) agency in digitally artifacted social interactions. It seeks to overcome the dual ontological approach to agency by observing its situated, multimodal, and dynamic enactment by interactants. The gathered body of research explores the notion of situated agency through various lenses. Indeed, contributions to this issue can be divided into four sections: human-chatbot interactions (Habscheid et al., 2023/this issue on smart speakers; Korbut, 2023/this issue on service telephone chatbots), human-robot interactions (Majlesi et al., 2023/this issue on a social robot called Furhat; Sormani & Hostettler 2023/this issue on robots in educational settings), interactions with digital technologies in specific contexts (Mlynář et al., 2023/this issue on autonomous cars and pedestrians, Klowait & Erofeeva, 2023/this issue on virtual reality and atypical users), and face-to-face human interactions involving the use of mobile devices (Robles et al., 2023/this issue).

2. Summary of the Articles

2.1 Situated agency in human-chatbot interactions

In the article by Habscheid, Hector and Hrncal, agency is not seen as something that certain ontological entities possess or do not possess in a stable way. Instead, the authors conceive of agency as a dynamic accomplishment, especially in voice-based exchanges with smart speakers. The achievement of agency is linked to local (linguistic) practices involving contributions from participants with unequal resources for participating. Following Hirschauer (2016), the authors distinguish between 'levels of activity' on an 'active-passive spectrum' and a 'proactive-inhibitive spectrum'. In this context, they empirically reconstruct how the smart speaker can appear in different situations. Finally, the article discusses the notion of agency in relation to the observed practices and a broader context of agency as described in media theory, including domestication theory as well as recent 'smart home'-technologies and platform logics.

In his paper, Korbut considers whether interactions with so-called 'conversational agents' (chatbots, voice assistants, etc.) can be viewed as a form of conversation. The author argues that such conversational agents are conversational in a proper sense. This conclusion is based on an analysis of the beginnings of 100 calls to a Russian municipal call centre handled by a chatbot. The identified features of inquiry formulations, silences, and overlaps at the beginning of the call indicate that users interact with the chatbot as a conversational partner instead of a voice interface. According to the author, for an interaction to qualify as a 'conversation', there only needs to be one co-participant (the weak participation requirement) who can understand all the turns of the interaction (the strong analysability requirement) as part of the ongoing conversation.

2.2 Situated agency in human-robot interactions

The study by Majlesi, Cumbal, Engwall, Gillet, Kunitz, Lymer, Norbby and Tuncer adresses agency by investigating turn-taking in human-robot interactions (HRI). Based on 15 video-recorded interactions between pairs of human participants and a social robot called Furhat, the authors examine how human participants perceive and handle violations in the normative order of turn-taking in social interaction. The authors show particular features of turn-taking with the robot and demonstrate how the robot may fail to respond to the human participants' bids to take a turn. In these sequences, the participants either complete the turn in progress and ignore the overlap caused by the robot's continuation of its turn, or they cut their turn short and restart in the next possible turn-transition place. These overlaps and failed smooth turn transitions are oriented to as accountable and interactionally problematic. The results of the study point to areas for improvement in robot engineering. They further illuminate routine projection practices and how human subjects orient toward normative expectations of ordinary social interactions, even when conversing with a robot.

In their paper, Sormani and Hostettler present a practice-based video analysis of student-robot interaction. The authors explicate the double interest of "provoking situations" as a praxeological topic and a pedagogical resource. The paper combines video analysis and a practical reenactment of two contrasting episodes of student-robot interaction. This approach allows to explicate (some of) the "tutorial problems" (Garfinkel, 2002) resulting from the practical reenactment, which recast and complement the video analysis. In particular, situated agency in student-robot interaction is revisited as a complex phenomenon and pedagogical issue. The paper also reflexively intervenes in ethnomethodology/conversation analysis as it prospects a (relatively) new avenue for EM/CA research. The authors articulate the practical reenactment of situated interaction as a heuristic strategy while leveraging the (arguably) "phenomenological difference" between EM and CA on education as a methodological resource. The authors conclude by discussing how and why the provocative impetus of science and technology studies (Woolgar, 2004) can, and perhaps should, be leveraged more broadly in prospective EM/CA studies.

2.3 Situated agency in interactions with digital technologies in specific contexts

The study by Mlynář, Eden and Evéquoz is based on an analysis of video recordings of a self-driving shuttle being tested as a means of public transportation. The paper analytically focuses on yielding, as achieved through pedestrians stepping aside, stopping, and letting the shuttle pass. The authors examine and describe how solitary pedestrians "stop aside" and how mobile formations of multiple persons take part in the practice. As "stopping aside" is a social action often followed by displays of gratitude, the authors also reflect on this facet regarding automated vehicles. According to the authors, agency reflexively emerges from the organised and sequential character of the situation. It is grounded in assemblages of human and technological aspects rather than originating in clearly distinguishable singular "actors" or "agents."

In their paper, Klowait and Erofeeva investigate how people with atypical bodily capabilities interact and overcome interactional challenges within virtual reality (VR). The authors demonstrate how non-speaking VR participants furnish their bodies, at-hand instruments, and interactive environment for practical purposes through a video-based multimodal analysis of a single case. Their findings contribute to renewed discussions of the relationship between agency and environment and the co-constructed nature of situated action. The authors also aim to contribute to the growing vocabulary of atypical interaction analysis and the broader context of ethnomethodological conceptualisations of unorthodox and fractured interactional ecologies.

2.4 Situated agency in face-to-face human interactions involving the use of digital devices

The study by Robles, DiDomenico, Raclaw and Joyce considers the role of agency in human interaction with mobile devices, particularly in the context of participants' quoting linguistic content from device screens. Using multimodal conversation analysis, the authors examine how mobile screen content is reproduced as locally relevant for updating information for co-present interlocutors. While informing-centred actions supported by mobile devices may appear to be an agentic intrusion into local interaction, the authors show that the organisation of device-accessed information and its meaning is nonetheless positioned in relation to how human participants animate device-supported updates into social action. More generally, this research contributes to knowledge about how device-related content is sequentially incorporated into face-to-face interaction.