Social Interaction

1. Introduction

The special issue on "Situated agency in digitally artifacted social interactions" (Ibnelkaïd & Avgustis, 2023) is a timely and highly important publication in studies of action and sense-making practices, with a focus on how materials can achieve agency within unfolding situations. Studying situated agency with an analytical sensitivity to how humans, nonhumans, materials, objects, nature and technologies assemble with humans in and through activities reflects a research interest omitted by language-centric or even human-centric (anthropocentric) analysis. It indicates an interest in unpacking multimodal actions, the senses or the bodies in intercorporeal interactions, which EMCA scholars in general deem highly important (e.g., Cekaite & Goodwin, 2021; Meyer et al., 2017; Mondada, 2021, 2022), but also emphasizes a stronger commitment to producing understandings of how exhibited meaning is indexically tied to material circumstances and an openness towards other-than-human forms of agency.

This is a welcome opportunity to reflect on some important topics relating to materials and agency. In this commentary, I will argue that we should expand the analytical sensitivity towards what I would call situated socio-material assemblages, based on a process philosophical discussion of the kind of ontology I believe we should explicate, and then adopt the methodological consequences. This approach builds on ethnomethodology, but in a way that shifts the focus from human members alone (anthropocentrism) to the situated activity as being assembled in and through emerging productions by a range of entities, the nature or relevance of which we cannot define before the situations emerge. I will propose the notion of assemmethodology as an ethnomethodological hybrid (in the vein of "technomethodology" (Crabtree, 2004; Dourish & Button, 1998)). The aim is to promote greater analytical sensitivity within EMCA¹ and related interactional approaches towards not just humans or their interactions with objects, but the ways in which activities are accomplished in and through local and specific forms of situated and distributed agency. I will return to this.

EMCA has clearly always been in opposition to the Cartesian mind-body distinction, for instance, by revealing cognition as situated, embodied, and tied to the ground (Maynard, 2006). Along the way, agency has been separated from dualistic conceptions, and yet the vast majority of EMCA papers nevertheless describe it as a solely human trait. Different positions strongly related to EMCA, such as Hutchins' (1995) distributed cognition, Goodwin's (2013) semiotics and activity systems, Suchman's (2007) situated action, and Enfield and Kockelman's (2017) distributed agency, have shown and argued how agency can be a situated property of nonhumans, and distributed across and within the situation. Other, more object-centered positions, such as those of Caronia (2018) and Malafouris (2019), take the opposite stance, and predetermine agency as a property of an entity. In actor-network theory, researchers are eager to follow the traces of objects and their agency through an endless network (Latour, 2005). The papers in this special issue show agency to be neither a solely human trait nor a trait predefined to objects, nor something that it is necessarily important to follow out of the situation. Rather, agency is seen as an emerging property of the situation and—in my interpretation—as a distributed achievement between the entities that currently form the assemblage.

One key reason for the pressing need to respecify agency is the current rapid development of AI. Advanced technologies, AI systems, robots, etc. all compel us to reconsider many aspects of the social world, not least agency, as shown in this special issue and in the growing body of other EMCA publications (Due, 2023; Pelikan et al., 2022). However, I wish to emphasize, firstly, that it is highly relevant to also include in these reflections of agency the studies of nonhumans within EMCA, for instance, horses (Kvart & Bowden, 2022), guide dogs (Due, 2021c), great apes (Mondada & Meguerditchian, 2022), and other animals (Mondémé, 2020). Secondly, and this is my main point, it is also important to broaden the analytical sensitivity to include not only interaction with nonhumans and objects, but also the emerging distributed production of activities in and through the ways in which entities assemble.

2. Papers in this Special Issue

The papers in this special issue show, in different ways, aspects of agency as a property of the situation. Robles, Raclaw, DiDomenico, and Joyce's study of human interaction with mobile devices shows how device-related content is sequentially incorporated into face-to-face interaction, and that the actions supported by mobile devices may sometimes have the character of an agentic intrusion into the local interaction, which is managed on a turn-by-turn basis. Habscheid, Moritz Hector, and Hrncal study voice-based exchanges using smart-speaker-technology and show how agency is a dynamic accomplishment bound to the local (linguistic) practices "carried out by" (or rather "involving contributions by") participants with unequal resources for participating. In his article, Korbut considers whether it is possible to view interactions with so-called "conversational agents" as a form of conversation, and argues that such agents are conversational, based on the criteria that a human treats the AI as a participant in the conversation. The study by Majlesi, Tuncer, Kunitz, Norrby, and Lymer deals with turn-taking in human-robot interactions (HRI) via a study of the social robot Furhat. They show how turn-transitions are oriented to as accountable and in some sense interactionally problematic and discuss the ways in which human subjects orient toward normative expectations of ordinary social interactions, even when conversing with a robot. They also analyze the robot's contribution, and thus its agency as an agentive entity in the emergence of trouble in HRI. In their article, Sormani and Hostettler present a practice-based video analysis of "student-robot" interaction in situ, and revisit agency in "student-robot" interaction as both an intricate phenomenon and a pedagogical issue. The article by Mlynář studies trials of self-driving shuttle buses as a means of public transportation. It shows how agency reflexively emerges from the organized and sequential character of the situation and is grounded in assemblages of human and technological aspects, rather than originating in clearly distinguishable singular "actors" or "agents." In their article, Klowait and Erofeeva investigate how people with atypical bodily capabilities interact within virtual reality (VR) and show how agency is a distributed achievement of multiple actors in the digital local environments that furnish the actors' capabilities.

In summary, the articles show, in different ways, and based on very different empirical settings involving humans and technologies, how agency is a property of the situation as it emerges, but also—although I would add that this is not explicitly the case in every paper—an essentially distributed achievement (Enfield & Kockelman, 2017). Collectively, these papers indicate the need to focus on the situation without applying an a priori understanding of who or what matters the most, but to focus on activities as inherently comprising the socio-material circumstances. In this commentary, the use of we reflects my intention to include research on situated social interaction, sense-making practices and the reciprocity of the socio-material world, in the tradition of context analysis (Birdwhistell, Scheflen, Kendon), Goffman, Garfinkel and Sacks—and the lines of research that followed from them—but also an attempt to steer the analytical sensitivity away from anthropocentrism.

In this commentary, I will briefly outline some of the ontological and epistemological considerations that could govern studies of not just situated agency, but the entire complex of situated socio-material assemblages, in ways that accommodate the situation, the activities, and its entities, rather than "just" the humans. I will suggest a future direction for the study of situated agency within complex situations, albeit without postulating that these are necessary steps. I am aware that the concept of socio-materiality (e.g., Cooren, 2020; Hawley, 2021; Leonardi, 2013; Orlikowski, 2007) lies on the border of a "classic" EMCA approach, but I also feel that researchers working under this umbrella term need to take our field forward even as we remain committed to a situated perspective on the process of real-world emergence. Even in the most basic turn-constructional units, meaning-making is inherently bound to the sequential context, but at the same time it is reflexively indexed by the socio-material environment of not just the bodied positions in space (e.g., Keevallik, 2013), but also the materials that afford the unfolding activities, as for instance shown in Goodwin's STS-oriented papers (Goodwin, 1993, 1995). However, although Goodwin remains a major source of inspiration for our research on the distributed nature of agency and materials, I do not think we necessarily need to be occupied with semiotics or cognition to understand the emerging assemblages. In the following, therefore, I will present some open-ended reflections aimed at further stimulating analytical sensitivity towards the components that make up social situations. My starting point for these reflections is ethnomethodology, and I will begin by briefly presenting what I consider to be the key aspects of studying situated agency.

3. Recognizing the Entanglement: Towards Assemmethodology

Modern science teaches us that the world is made up of matter, all the way down to the atomic level. There is an interaction, an interconnectedness, an entanglement or a relationship between everything in the world. The I is not just related to the Thou (Buber, 2012), but potentially everything in and between the I and the Thou. Materials are not add-ons or something with which people occasionally interact but are inherently interwoven into human existence. This is the case not only for social interaction, but all aspects of the universe—the visible and invisible, the alive and the dead. In the work of Whitehead (1979) and Barad (2007), among others, we find interpretations of the modern physics of Einstein and Bohr, leading to a social ontology that can be useful for achieving a broadened understanding of that which constitutes the world in which we live and act.

In this deeper sense, there can be no actions in and of themselves, only inter-actions and entanglement—as every action in the world affects something else, in a constant process of becoming. At the very least, the air surrounding the action is affected—it moves when I make a gesture, when I breathe, when I speak, etc. I do not suggest that we begin to study interactions with the air as a program per se; rather I am laying out aspects of an ontology. But then again, why not study the way the wind is observably part of the accomplishment of an emerging assemble? For instance, when we are pushed by a strong wind, we adjust our bodily position, which can in turn affect other courses of action. Interactions involve not only nonhumans, technologies, or materials—nature itself is full of strong forces with which we interact and that form important parts of emerging assemblages. The capacity to act and the capacity to substantially influence the ongoing social world is not an exclusively human trait. Many types of agents, including nonhumans, materials, and nature, can produce and have an effect on observable actions.

Latour (2005) and others in material semiotics and actor-network theory (e.g., Law, 2004) adhere to the same basic relational ontology of interconnectedness, as do researchers based in ecological thinking (Ingold, 2000, 2012), among others. However, there are two essential differences between these kinds of positions and the empirical program that relates to EMCA: 1) ANT and ecological thinking theory tend to allow the analysis to be based on the analyst's interpretations and theoretical constructs, rather than studying emergence as an endogenous production; and 2) they tend to follow the traces wherever they lead, instead of staying within the situation in which people and stuff assemble and sense-making occurs.

In the work of Deleuze and Guattari, we find a theoretical conception of assemblages that might inform our thinking, and here a Garfinkelian praxeologial "misreading" may be illuminating (c.f. Eisenmann & Lynch, 2021). Both Deleuze and Guattari (2004) describe assemblages as temporary productions that can be separated again, like a driver-car (Dant, 2004) or a robot-doctor (Due, 2021b). Whereas Deleuze and Guattari's description of assemblages is wild and endless, like Latour's networks, a praxeological respecification can situate assemblages as temporarily practical accomplishments within situations, as being made by the entities that configure those situations. Based on ethnomethodology, and what Garfinkel called the "rule of practical circumstances" (Garfinkel, 1967, p. 74), we should base the analysis only on the situation, and not stipulate any theoretical concepts a priori concerning, for instance, who or what possesses agency. Instead, the analysis leads to a posteriori findings.

This boils down to an existential, phenomenological concern for humans, but also entails an analytical sensitivity to the assemblages and their methodic production, that is, their assem-methodology. To propose the concept of assemmethodology is to suggest taking the principles from ethnomethodology concerning the process of philosophical ontology, for instance, a world in constant becoming "for another first time" (Garfinkel, 1967, p. 10), and to focus on methodic, orderly productions of situated activities and the ways in which agency per definition is situated and distributed. I will return to this shortly.

As we know, Garfinkel was primarily interested not in humans as such, but in the "organized activities of everyday life" (Garfinkel, 1967, p. vii). Among other terms, he used cohort to describe the coming together of various elements into a recognizable phenomenon, such as a freeway traffic flow (Garfinkel, 1996) or an assembly line (Garfinkel & Livingston, 2003).² Traffic is made up of drivers, cars, roads and all the rest collectively "making traffic together" as a lived production, being just this or that. Although traces of assemblages are found in Garfinkel's writing, the scholarly work of analytically processing these concepts has not expanded to include an analytical sensitivity towards the activity as an assemblage of various heterogenous materials, but rather has manifested in studies of orderly interactions between units (e.g., Ivarsson & Greiffenhagen, 2015; Laurier et al., 2020). While these studies of cohorts made up of materials are perfectly in line with Garfinkel's writings, I think we need to expand further, and pay more attention to not just the objects that are interacted with, but also the materials that essentially afford, structure, affect and become part of the situation and the distributed agency of the activity.

4. An Existential Concern with Humans: The Phenomenological Heritage

Ethnomethodology's goal has never been to study persons "holistically", but to study members (folks/ethnos) by focusing on the skilled, competent, and practical ways in which their activities are accomplished (Have, 2002). Nonetheless, especially in CA, there has still been a strong emphasis on humans and human resources alone. Surely, nonhuman interaction and sociality, for instance, among baboons, can be studied using CA turn-taking terminology (Mondada & Meguerditchian, 2022), with a view to also using such studies to enhance understanding of human sociality. This can also be the case when people are producing understanding in interaction with AI, as shown in this special issue. In other words, when we are studying human sociality, we are not restricted to just studying humans. It is reasonable to imagine situations in which concepts such as sequential organization, turn-taking, intersubjectivity and the orderly production of phenomenal fields may be applied to pure machine-machine interactionn—unless we predefine these as exclusively human terms rather than technical descriptions. The rapid development we are now witnessing in generative AI like ChatGPT will compel us to revisit linguistic meaning-making and understanding as other-than-human productions.

Surely, the underlying intentionality and designed affordances³ of technologies and other manmade objects enable them to configure in certain ways in situ. An ANT analysis can reveal such forms of distributed agency between developers and practices. However, staying within the situation, observing how two machines or two animals interact, take turns, and produce recognizable actions, without any humans being part of the situation, is in principle doable using EMCA terminology and methodology.

The initiation or production of an action does not require the presence of a human. It only requires an "action-able" (Due, 2021a) agent, which can be an animal or another part of nature, or certain kinds of objects or technologies. Materials that are incapable of producing any action on their own, for instance, a hammer, can still potentially, in and through their affordances and sensory capacities (being seen, being touched), structure activities within the phenomenal field, as Hutchins (1995) and several of Goodwin's studies have shown (2007).

The—perhaps radical—point of view is then that there is no sharp distinction between humans and nonhumans, and no a priori definition of agency. There is only a) a dogmatic focus on the emerging and demarcated situation; and b) an existential concern with what we can learn about human sociality from a study.

The fundamental question remains existential, in the phenomenological tradition from Husserl, Heidegger, Merleau-Ponty, Schutz, and Gurwitsch (who we know Garfinkel "misread" in great detail). By "existential" I do not directly refer to existentialist philosophy in the tradition of, for instance, Kierkegaard, Sartre or Nietzsche, which concerns the introspective examination of the subjective problem of meaning and the struggle with absurdism, but rather the basic question of what it means to be human, viewed through a lens of actions, practices and experience as exhibited. Surely, we can learn about what it means to be human by examining the observable production of actions accomplished by other types of agents.

A praxeological respecification of Heidegger shows up in the form of questions concerning what it means, in practical terms, to be a "being-in-the-world", as "always already" (immer schon) (Heidegger, 2010), as flesh and blood in a world of existing things (materials, nature, etc.). It is about what it means not only to interact with the world, but to be part of it, to be entangled and assembled. The question, then, is how to study situations with an existential concern for humans, while still abandoning anthropocentrism as a methodology. In other words, what can we learn about human sociality from studying the organized activities of everyday life as they show up as organized assemblages in the context of interspecies or other forms of inter-agent interactions in complex socio-material situations? What do we learn about human sociality when our starting point is not language or even human action, but the phenomenal field and its gestalt contexture?

This leads to methodological questions about what it means for something to be observable and recognizable, and what relevance means. For some language-centric CA studies, these will not be tough questions, because such studies are grounded in linguistically produced sequences and observability, and therefore relevance can be tracked using the "next-turn proof procedure" (Sacks et al., 1974). However, when expanding from the relative neatness of talk-in-interaction and moving towards the messiness of bodies and materials in the world, we also need to reconsider the relevance of the "next-turn proof procedure" as an analytical principle, and perhaps return to deeper reflections on time, simultaneity, emergence, and consequentiality. In this way, we can adopt an analytical sensitivity to the nature of causes and effects and how they are observable via relationships—some as displayed actions, others as features of situations that are part of the ongoing production of the current situated reality.

5. Emergence and Situation

Garfinkel was primarily concerned with the problem of social order, and he dealt with it partly based on the ontological fact that everything happens "for another first time" (Garfinkel, 1967, p. 10). There are two important aspects to this "slogan" that we must take seriously.

Firstly, although this makes for a nice, succinct slogan, it entails a whole complex ontology that sometimes gets lost in "constructive analysis" (Button et al., 2022). It is a very concrete fact that each action and each situation has never happened before and will never happen again in exactly the same way. Everything happens for the first time in exactly this or that way. We can imagine all sorts of things, but we cannot imagine precisely what will happen next in all its detail. This leads us to another Garfinkel quote, namely that society "cannot be imagined but is only actually found out, and just in any actual case" (Garfinkel, 1996, p. 8). This concerns the process-philosophical point that everything is in the making all the time, which requires us to study assemblages as particular forms of thisness (haecceity) (Garfinkel, 1991). Here, there is a strong crossover with process philosophy, which has yet to be fully worked out (but see Llewellyn, 2014). Garfinkel assuredly knew of Heraclitus, Spinoza, Leibniz, among others (Helin et al., 2014), and he read Whitehead, but only minimally referred to him (Garfinkel, 1988, p. 105). He most likely approached Whitehead via the same lens of misreading and praxeological respecification that he used for the phenomenologists Husserl, Heidegger, Merleau-Ponty, and Gurwitsch. Studying the distribution of agency within assemblages thus requires minute attention towards its constantly emerging character—it is a matter of sequences, of course, but more profoundly, it is about time, temporality, and progressivity.

The other point relating to the famous slogan ("for another first time") relates to the situation as a distinct phenomenal field. The emergence and flow of actions made up of heterogenous elements that come together exist only within this or that situation. Although the elements that go into making this or that situation expand throughout endless networks, we may choose to study them as they come together and occur in temporary assemblages within the situation, as defined in and through practical actions by the members themselves in situ. It is not the analyst's job to demarcate the situation as a phenomenal field, but to unpack it as a production done by members, that is, any agents, along with their distribution of agency within the situation. Further, situations are obviously not just face-to-face encounters, in Goffman's (1964) or Kendon's (1976) original sense, but can also occur as digitally mediated and dislocated forms.

In sum, focusing on situated socio-material assemblages compel us to take seriously the problems of reification and system building as manifest representations of a world that is constantly in flux, in becoming and in which assemblages is a lived work of coming together and being separated. It requires of us to stick with emerging things and situations rather than producing stable worlds.

6. Agency and Assemmethodology

Studying socio-materiality and situated and distributed agency in the tradition of ethnomethodology requires constantly taking an emic approach to how the empirical world is in a process of becoming from within empirical situations themselves as endogenous productions. Much can be learned from how ethnomethodology developed within science and technology studies (Lynch, 1993, 2017; Suchman, 2007), but that field has unfortunately been unable to include studies of socio-material assemblages that do not study science in the making per se.

Based on the papers in this special issue, along with others that have recently taken up questions of agency, my aim in this commentary is to encourage a greater analytical sensitivity to the emerging situations as they are endogenously produced, without any predefined, deterministic views on what matters, who matters, when something matters or, indeed, what it means to "matter".

This is obviously not the same as stating that humans and objects are equal (cf. the misunderstanding of "generalized symmetry" in ANT) or have the same capacities. Rather it is about recognizing that humans and "the rest", together in situ, have the capacity to act. While nonhuman animals and materials such as computer systems actively and autonomously produce actions, others with no autonomous capacities, for instance, a hammer, can become part of the assemblage from their very presence within the sensorially accessible situation—they are visible, touchable, smellable, hearable and thus part of the situation. Accordingly, it is not necessarily always the case that we should focus on how people interact with objects or how people use multimodal resources. Rather, it is perhaps a question of how any current activity made up of heterogeneous elements within assemblages as a haecceity comes into being, with a greater analytical sensitivity towards the consequences of materials for this or that situation, purely by being there. By activity, I simply mean situations in which things are happening and actions are produced, and where the activity is itself accountable as being something, for instance, having a meal, partaking in a meeting, conversing with friends, etc. Animals—and perhaps also AI systems—can define and negotiate activities as being something, for instance, a serious fight or joyous play (cf. Bateson's (1967) notion of metacommunication). In other words, neither activities nor actions are restricted to humans. While there are obviously observable differences in the types of activities and actions that can be accomplished and negotiated, in principle there is no sharp distinction.

In this commentary, I have tried to suggest a version of ethnomethodology that maintains a strong focus on organized activities, without a priori determining that only humans possess agency, or that we are required to understand situations exclusively in terms of human production. I think this is well grounded in ethnomethodology. However, Garfinkel was primarily interested in humans as members, and in their methods for making activities "visibly-rational-and-reportable-for-all-practical-purposes, that is, ‘accountable,' as organizations of commonplace everyday activities" (Garfinkel, 1967, p. vii). By proposing the hybrid assemmethodology, I also suggest that reflexivity and accountability are not just related to human conduct in isolation, but to the observable consequentiality of other types of actions from other types of agents, and the ways in which they are interwoven and entangled with human sociality.

We know from Newton's Third Law that one movement or action affects another (the principle of action and reaction). The laws of physics are obviously not a topic for ethnomethodology or EMCA research in itself (except as a scientific production in the lab). However, as an ontological fact, it is important to recognize the basic character of responsiveness as also being an other-than-human trait. The question is, where are the limits or boundaries for what we understand as observable and reflexively relevant within situations? This question will surely receive different answers within our community. I am just suggesting that those of our research papers that deal with the socio-material complexity of situations, also incorporate a stronger analytical sensitivity towards situated agency, materials, object, technologies, nature, animals, etc. as being assembled in situ.

^{1 In this article, I differentiate between ethnomethodology (EM), conversation analysis (CA) and ethnomethodological conversation analysis (EMCA/ethnoCA). In short, and as highly rigid forms, EM is the study of order with a radical view towards members' productions and a resistance towards theory-building; CA is the study of sequential organizations and interaction, often with an interest in building collections and even theories or programs. EMCA is then an interest in members' productions of order, with a focus on the ways in which they are observable in social interaction. ↩}