Distributed Cognition in Fractured Ecologies

Collaborative Problem-Solving in Video-Mediated Interaction





Real-world problem solving, video-mediated interaction, computer-mediated communication, distributed cognition


In this article we present a conversation analytic case study of a video-mediated teleconsultation in which the participants face a problem with the audio connection. During the problem-solving process the interactants need to direct one another’s attention and action in relation to technological artefacts to solve the problem. Video mediation limits physical access to distant participants’ physical ecology, which is overcome by fitting interactional practices to the communicative medium available at a given moment. Drawing on insights from the distributed cognition perspective and the CA perspective on participation, multimodality, and epistemics, we propose specifications to existing theories of problems solving, seeking to develop integrative approaches to real-world problem solving.


Aagaard, J. (2021). 4E cognition and the dogma of harmony. Philosophical Psychology, 34(2), 165–181. https://doi.org/10.1080/09515089.2020.1845640

Arminen, I., Licoppe, C., & Spagnolli, A. (2016). Respecifying mediated interaction. Research on Language and Social Interaction, 49(4), 290–309. https://doi.org/10.1080/08351813.2016.1234614

Arminen, I., & Poikus, P. (2009). Diagnostic reasoning in the use of travel management system. Computer Supported Cooperative Work, 18, 251–276. https://doi.org/10.1007/s10606-008-9086-3

Balaman, U. (2021). The interactional organization of video-mediated collaborative writing: Focus on repair practices. TESOL Quarterly, 5(3), 979–993. https://doi.org/10.1002/tesq.3034

Bowden, H. M. (2019). Problem-solving in collaborative game design practices: Epistemic stance, affect, and engagement. Learning, Media and Technology, 44(2), 124–143. https://doi.org/10.1080/17439884.2018.1563106

Bjørndahl, J. S., Fusaroli, R., Østergaard, S., & Tylén, K. (2014). Thinking together with material representations: Joint epistemic actions in creative problem solving. Cognitive Semiotics, 7(1), 103–123. https://doi.org/10.1515/cogsem-2014-0006

Button, G. (2008) Against ‘Distributed Cognition’. Theory, Culture and Society, 25(2), https://doi.org/10.1177/0263276407086792

Carr, C. T. (2020). CMC is dead, long live CMC! Situating computer-mediated communication scholarship beyond the digital age. Journal of Computer-Mediated Communication, 25(1), 9–22. https://doi.org/10.1093/jcmc/zmz018

Clark, A., & Chalmers, D. (1998). The extended mind. Analysis, 58(1), 7–19. https://doi.org/10.1093/analys/58.1.7

Couper-Kuhlen, E. (2014). What does grammar tell us about action? Pragmatics, 24(3), 623–647. https://doi.org/:10.1075/prag.24.3.08cou

Cowley, S. J., & Vallée-Tourangeau, F. (Eds.). (2013). Cognition beyond the brain: Computation, interactivity and human artifice. Springer.

Due, B. L. (2016). Fælles orientering som ressource for idéudvikling: En single case-analyse baseret på distributed cognition (DC) conversation analysis (CA) [Shared orientation as a resource for idea development: A single case analysis based on distributed cognition (DC) conversation analysis (CA)]. Nydanske Sprogstudier NyS, 50, 86–119. https://doi.org/10.7146/nys.v1i50.23799

Due, B. L., & Lange, S. B. (2020). Body part highlighting: Exploring two types of embodied practices in two sub-types of showing sequences in video-mediated consultations. Social Interaction: Video-Based Studies of Human Sociality, 3(3). https://doi.org/10.7146/si.v3i3.122250

Due, B. L., Lange, S. B., Femø Nielsen, M., & Jalskov, C. (2019). Mimicable embodied demonstration in a decomposed sequence: Two aspects of recipient design in professionals' video-mediated encounters. Journal of Pragmatics, 152, 13–27. https://doi.org/10.1016/j.pragma.2019.07.015

Due, B. L., & Licoppe, C. (2020). Video-mediated interaction (VMI): Introduction to a special issue on the multimodal accomplishment of VMI institutional activities. Social Interaction: Video-Based Studies of Human Sociality, 3(3). https://doi.org/10.7146/si.v3i3.123836

Flanagin, A. J. (2020). The conduct and consequence of research on digital communication. Journal of Computer-Mediated Communication, 25(1), 23–31. https://doi.org/10.1093/jcmc/zmz019

Gibson, J. J. (1979). The ecological approach to visual perception. Psychology Press.

Gigerenzer, G., & Todd, P. M. (2001). Simple heuristics that make us smart. Oxford University Press.

Goodwin, C. (1980). Restarts, pauses, and the achievement of a state of mutual gaze at turn-beginning. Sociological Inquiry, 50(3–4), 272–302. https://doi.org/10.1111/j.1475-682X.1980.tb00023.x

Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606–633. https://doi.org/10.1525/aa.1994.96.3.02a00100

Goodwin, C. (2013). The co-operative, transformative organization of human action and knowledge. Journal of Pragmatics, 46(1), 823. https://doi.org/10.1016/j.pragma.2012.09.003

Goodwin, C. (2017). Co-operative action. Cambridge University Press.

Goodwin, C., & Goodwin, M. H. (2004). Participation. In A. Duranti (Ed.), A companion to linguistic anthropology (pp. 222–244). Blackwell. https://doi.org/10.1002/9780470996522.ch10

Grigorov, D. N., & Snoeck Henkemans, A. F. (2019). Hypothetical questions as strategic devices in negotiation. Negotiation Journal, 35, 363–385. https://doi.org/10.1111/nejo.12297

Hakulinen, A., & Seppänen, E. L. (1992). Finnish kato: From verb to particle. Journal of Pragmatics, 18(6), 527–549. https://doi.org/10.1016/0378-2166(92)90118-U

Hansen, J. P .B. (2020). Invisible participants in a visual ecology: Visual space as a resource for organising video-mediated interpreting in hospital encounters. Social Interaction: Video-Based Studies of Human Sociality, 3(3). https://doi.org/10.7146/si.v3i3.122609

Heath, C., & Luff, P. (1992). Collaboration and Control Crisis Management and Multimedia Technology in London Underground Line Control Rooms. Computer Supported Cooperative Work, 1, 69–94. https://doi.org/10.1007/BF00752451

Heritage, J. (1984). A change-of-state token and aspects of its sequential placement. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action: Studies in conversation analysis (pp. 299–345). Cambridge University Press. https://doi.org/10.1017/CBO9780511665868.020

Heritage, J. (2012). Epistemics in conversation. In J. Sidnell & T. Stivers (Eds.), The handbook of conversation analysis (pp. 370–394). Wiley-Blackwell. https://doi.org/10.1002/9781118325001.ch18

Heritage, J., & Raymond, G. (2012). Navigating epistemic landscapes: Acquiescence, agency and resistance in responses to polar questions. In J. De Ruiter (Ed.), Questions: Formal, functional and interactional perspectives (pp. 179–192). Cambridge University Press. https://doi.org/10.1017/CBO9781139045414.013

Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition: toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction, 7(2), 174–196. https://doi.org/10.1145/353485.353487

Hutchby, I. (2001). Conversation and technology: From the telephone to the internet. Polity Press.

Hutchins, E. (1980). Culture and inference. Harvard University Press.

Hutchins, E. (1995). Cognition in the wild. MIT Press.

Hutchins, E. (2011). Enculturating the supersized mind. Philosophical Studies, 152, 437–466. https://doi.org/10.1007/s11098-010-9599-8

Ilomäki, S., & Ruusuvuori, J., (2022). Preserving client autonomy when guiding medicine taking in telehomecare: A conversation analytic case study. Nursing Ethics. https://doi.org/10.1177%2F09697330211051004

Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In G. H. Lerner (Ed.), Conversation analysis: Studies from the first generation (pp. 13–23). John Benjamins. https://doi.org/10.1075/pbns.125.02jef

Järvilehto, T. (2009). The theory of the organism-environment system as a basis of experimental work in psychology. Ecological Psychology, 21(2), 112–120. https://doi.org/10.1080/10407410902877066

Koffka, K. (1935). Principles of Gestalt psychology. Harcourt, Brace and Company.

Luff, P., Heath, C., Kuzukoka, H., Hindmarsh, J., Yamazaki, K., & Oyama, S. (2003). Fractured ecologies: Creating environments for collaboration. Human-Computer Interaction, 18(1), 51–84. https://doi.org/10.1207/S15327051HCI1812_3

Luff, P., Heath, C., Yamashita, N., Kuzuoka, H., & Jirotka, M. (2016). Embedded reference: Translocating gestures in video-mediated interaction. Research on Language And Social Interaction, 49(4), 342–361. https://doi.org/10.1080/08351813.2016.1199088

Mlynář, J., & Arminen, I. (2023). Respecifying social change: the obsolescence of practices and the transience of technology. Frontiers in Sociology, 8(1222734). https://doi.org/10.3389/fsoc.2023.1222734

Mlynár, J., González-Martínez, E., & Lalanne, D. (2018). Situated organization of video-mediated interaction: A review of ethnomethodological and conversation analytic studies. Interacting with Computers, 30(2), 73–84. https://doi.org/10.1093/iwc/iwx019

Mondada, L. (2019). Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics, 145, 47–62. https://doi.org/10.1016/j.pragma.2019.01.016

Oittinen, T. (2018). Multimodal accomplishment of alignment and affiliation in the local space of distant meetings. Culture and Organization, 24(1), 31–53. https://doi.org/10.1080/14759551.2017.1386189.

Oittinen, T. (2020). Noticing-prefaced recoveries of the interactional space in a video-mediated business meeting. Social Interaction: Video-Based Studies of Human Sociality, 3(3). https://10.7146/si.v3i3.122781.

Oittinen, T. (2023). Including Written Turns in Spoken Interaction: Chat as an Organizational and Participatory Resource in Video-Mediated Activities. Research on Language and Social Interaction, 56(4), 269–290. https://doi.org/10.1080/08351813.2023.2272524.

Olbertz-Siitonen, M. (2015). Transmission delay in technology-mediated interaction at work. PsychNology Journal, 13(2–3), 203–234. https://jyx.jyu.fi/handle/123456789/51325

Raymond, C. W., & Heritage, J. (2021). Probability and valence: Two preferences in the design of polar questions and their management. Research on Language and Social Interaction, 54(1), 60–79. https://doi.org/10.1080/08351813.2020.1864156

Raymond, G. (2003). Grammar and social organization: Yes/no interrogatives and the structure of responding. American Sociological Review, 68(6), 939–967. https://doi.org/10.2307/1519752

Rintel, S. (2013a). Tech-tied or tongue-tied? Technological versus social trouble in relational video calling. In Ralph E. Sprague (Ed.), Proceedings of the Forty-Sixth Hawaii International Conference on System Sciences, 3343–3352. IEEE Computer Society. http://dx.doi.org/10.1109/HICSS.2013.512

Rintel, S. (2013b). Video calling in long-distance relationships: The opportunistic use of audio/video distortions as a relational resource. The Electronic Journal of Communication / La Revue Électronique de Communication, 23(1–2). https://cios.org/EJCPUBLIC/023/1/023123.HTML

Rintel, S. (2015). Omnirelevance in technologized interaction: Couples coping with video calling distortions. In R. Fitzgerald & W. Housley (Eds.), Advances in Membership Categorisation Analysis (pp. 123–150). SAGE.

Ruhleder, K., & Jordan, B. (2001). Co-constructing non-mutual realities: Delay-generated trouble in distributed interaction. Computer Supported Cooperative Work, 10, 113–138. https://doi.org/10.1023/A:1011243905593

Saatçi, B., Akyüz, K., Rintel, S., & Nylandsted Klokmose, C. (2020). (Re)Configuring hybrid meetings: Moving from user-centered design to meeting-centered design. Computer Supported Cooperative Work, 29, 769–794. https://doi.org/10.1007/s10606-020-09385-x

Sarathy, V. (2018). Real world problem-solving. Frontiers in Human Neuroscience, 12(261). https://doi.org/10.3389/fnhum.2018.00261

Schegloff, E. A. (1987). Analysing single episodes of interaction: An exercise in conversation analysis. Social Psychology Quarterly, 50(2), 101–114. https://doi.org/10.2307/2786745

Schegloff, E. A. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge University Press.

Seuren, L. M., Wherton, J., Greenhalgh, T., Cameron, D., A’Court, C., & Shaw, S. E. (2020). Physical examinations via video for patients with heart failure: Qualitative study using conversation analysis. Journal of Medical Internet Research, 22(2). https://doi.org/10.2196/16694

Shaw, S. E., Seuren, L. M., Wherton, J., Cameron, D., A’Court, C., Vijayaraghavan, S., Morris, J., Bhattacharya, S., & Greenhalgh, T. (2020). Video consultations between patients and clinicians in diabetes, cancer, and heart failure services: Linguistic ethnographic study of video-mediated interaction. Journal of Medical Internet Research, 22(5). https://doi.org/10.2196/18378

Sidnell, J., & Stivers, T. (Eds.). (2013). The handbook of conversation analysis. Wiley-Blackwell.

Speer, S. A. (2012). Hypothetical questions: A comparative analysis and implications for “applied” vs. “basic” conversation analysis. Research on Language and Social Interaction, 45(4), 352–374. https://doi.org/10.1080/08351813.2012.724987

Stasser, G., & Abele, S. (2020). Collective choice, collaboration, and communication. Annual Review of Psychology, 71, 589–612. https://doi.org/10.1146/annurev-psych-010418-103211

Steffensen, S. V. (2009). Language, languaging and the extended mind hypothesis. Pragmatics & Cognition, 17, 677–697. https://doi.org/10.1075/pc.17.3.10ste

Steffensen, S. V. (2013). Human interactivity: Problem-solving, solution-probing and verbal patterns in the wild. In S. J. Cowley & F. Vallée-Tourangeau (Eds.), Cognition beyond the brain: Computation, interactivity and human artifice (pp. 195–221). Springer. https://doi.org/10.1007/978-1-4471-5125-8_11

Stevanovic, M., & Peräkylä, A. (2012). Deontic authority in interaction: The right to announce, propose, and decide. Research on Language & Social Interaction, 45(3), 297–321. https://doi.org/10.1080/08351813.2012.699260

Stivers, T. & Rossano, F. (2010). Mobilizing Response. Research on Language and Social Interaction, 43(1), 3–31. https://doi.org/10.1080/08351810903471258

Stivers, T. & Sidnell, J. (2016). Proposals for activity collaboration. Research on Language and Social Interaction, 49(2), 148–166. https://doi.org/10.1080/08351813.2016.1164409

Stommel, W., Licoppe, C., & Stommel, M. (2020). “Difficult to assess in this manner”: An “ineffective” showing sequence in post-surgery video consultation. Social Interaction: Video-Based Studies of Human Sociality, 3(3). https://doi.org/10.7146/si.v3i3.122581

Stommel, W. J. P., & Stommel, M. W. J. (2021). Participation of companions in video-mediated medical consultations: A microanalysis. In J. Meredith, D. Giles, & W. Stommel (Eds.), Analysing digital interaction (pp. 177–203). Springer International. https://doi.org/10.1007/978-3-030-64922-7_9

Tarasmundi, S. B., & Linell, P. (2017). Insights and their emergence in everyday practices: The interplay between problems and solutions in emergency medicine. Pragmatics & Cognition, 24(1), 62–90. https://doi.org/10.1075/pc.17002.tra

Vallée-Tourangeau, F. (2014). Insight, interactivity and materiality. Pragmatics & Cognition, 22(1), 27–44. https://doi.org/10.1075/pc.22.1.02val

VISK. (2004). Iso suomen kielioppi [The comprehensive Finnish grammar]. Hakulinen, A., Vilkuna, M., Korhonen, R., Koivisto, V., Heinonen, T.-R., & Alho, I. (Eds). Finnish Literature Society. http://scripta.kotus.fi/visk

Wertheimer, M. (1982). Productive thinking (Enlarged ed.). University of Chicago Press.

Weisberg, R. W. (2015). Toward an integrated theory of insight in problem solving. Thinking & Reasoning, 21(1), 5–39. https://doi.org/10.1080/13546783.2014.886625

Zhang, J., & Patel, V. L. (2006). Distributed cognition, representation, and affordance. Pragmatics & Cognition, 14(2), 333–341. https://doi.org/10.1075/pc.14.2.12zha

Öllinger, M., & Goel, V. (2010). Problem solving. In B. M. Galtzeder, V. Goel, & A. von Müller (Eds.), Towards a theory of thinking: Building blocks for a conceptual framework (pp. 3–21). Springer. https://doi.org/10.1007/978-3-642-03129-8_1




How to Cite

Ilomäki, S., & Stevanovic, M. (2024). Distributed Cognition in Fractured Ecologies: Collaborative Problem-Solving in Video-Mediated Interaction. Social Interaction. Video-Based Studies of Human Sociality, 7(1). https://doi.org/10.7146/si.v7i1.132207