When a Robot Comes to Life
The Interactional Achievement of Agency as a Transient Phenomenon
DOI:
https://doi.org/10.7146/si.v5i3.129915Keywords:
human-robot interaction, social robotics, agency, participation, sequence organization, autonomyAbstract
Conceptualizing agency is a long-standing theoretical concern. Taking an ethnomethodological and conversation analytic perspective, we explore agency as the oriented to capacity to produce situationally and sequentially relevant action. Drawing on video recordings of families interacting with the Cozmo toy robot, we present a multimodal analysis of a single episode featuring a variety of rapidly interchanging forms of robotic (non-)agency. We demonstrate how agency is ongoingly constituted in situated interaction between humans and a robot. Describing different ways in which the robot’s statuses as either an agent or an object are interactionally embodied into being, we distinguish “autonomous” agency, hybrid agency, ascribed agency, potential agency and non-agency.
References
Alač, M. (2016). Social robots: Things or agents? AI & SOCIETY, 31(4), 519–535. https://doi.org/10.1007/s00146-015-0631-6
Alač, M., Movellan, J., & Tanaka, F. (2011). When a robot is social: Spatial arrangements and multimodal semiotic engagement in the practice of social robotics. Social Studies of Science, 41(6), 893–926. https://doi.org/10.1177/0306312711420565
Antaki, C., & Crompton, R. J. (2015). Conversational practices promoting a discourse of agency for adults with intellectual disabilities. Discourse & Society, 26(6), 645–661. https://doi.org/10.1177/0957926515592774
Asaro, P. M. (2000). Transforming society by transforming technology: the science and politics of participatory design. Accounting, Management and Information Technologies, 10(4), 257–290. https://doi.org/https://doi.org/10.1016/S0959-8022(00)00004-7
Auer, P., Bauer, A., & Hörmeyer, I. (2020). How Can the ‘Autonomous Speaker’ Survive in Atypical Interaction? The Case of Anarthria and Aphasia. In R. Wilkinson, J. P. Rae, & G. Rasmussen (Eds.), Atypical Interaction: The Impact of Communicative Impairments within Everyday Talk (pp. 373–408). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-28799-3_13
Axelsson, A., Buschmeier, H., & Skantze, G. (2022). Modeling Feedback in Interaction With Conversational Agents—A Review. Frontiers in Computer Science, 4. https://doi.org/10.3389/fcomp.2022.744574
Bekey, G. A. (2005). Autonomous Robots: From Biological Inspiration to Implementation and Control. MIT Press.
Bohus, D., & Horvitz, E. (2011). Multiparty Turn Taking in Situated Dialog: Study, Lessons, and Directions. In Proceedings of the SIGDIAL 2011 Conference (pp. 98–109). USA: Association for Computational Linguistics.
Boudouraki, A., Fischer, J. E., Reeves, S., & Rintel, S. (2021). “I can’t get round” Recruiting assistance in mobile robotic telepresence. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3), 1–21. https://doi.org/10.1145/3432947
Bradshaw, J. M., Hoffman, R. R., Johnson, M., & Woods, D. D. (2013). The Seven Deadly Myths of “Autonomous Systems.” IEEE Intelligent Systems, 28(3), 54–61. https://doi.org/10.1109/MIS.2013.70
Braitenberg, V. (1986). Vehicles. Experiments in Synthetic Psychology. MIT Press.
Bratman, M.E., 1987, Intention, Plans, and Practical Reason, Cambridge, MA: Harvard University Press.
Breazeal, C. (2002). Designing Sociable Robots. Cambridge, MA: MIT Press.
Broth, M., Cromdal, J., & Levin, L. (2019). Telling the other’s side: Formulating others’ mental states in driver training. Language & Communication, 65, 7–21. https://doi.org/10.1016/j.langcom.2018.04.007
Button, G., & Sharrock, W. W. (2010). The structure problem in the context of structure and agency controversies. In A. Denis & P. J. Martin (Eds.), Human Agents and Social Structures (pp. 17–33). Manchester: Manchester University Press.
Bødker, S., Dindler, C., Iversen, O. S., & Smith, R. C. (2021). Participatory Design. Synthesis Lectures on Human-Centered Informatics, 14(5), i–143. https://doi.org/10.2200/S01136ED1V01Y202110HCI052
Clark, H. H., & Fischer, K. (2022). Social robots as depictions of social agents. Behavioral and Brain Sciences, 1–33. https://doi.org/10.1017/S0140525X22000668
Clayman, S. E. (2013). Agency in response: The role of prefatory address terms. Journal of Pragmatics, 57, 290–302. https://doi.org/10.1016/j.pragma.2012.12.001
Coleridge, S. T. (1817/2004). Biographia Literaria. https://www.gutenberg.org/files/6081/6081-h/6081-h.htm
Coulter, J. (1979). The social construction of mind: Studies in ethnomethodology and linguistic philosophy. New Jersey: Bowman & Littlefield.
Curl, T. S. (2006). Offers of assistance: Constraints on syntactic design. Journal of Pragmatics, 38(8), 1257–1280. https://doi.org/10.1016/j.pragma.2005.09.004
Davidson, D., 1980, Essays on Actions and Events, Oxford: Clarendon Press.
Defense Science Board. (2012). The role of autonomy in DoD Systems (Task Force Report No. ADA566864). Department of Defense United States of America. https://dsb.cto.mil/reports/2010s/AutonomyReport.pdf
Demuth, C. (2021). Managing accountability of children’s bodily conduct: Embodied discursive practices in preschool. In S. Wiggins & K. O. Cromdal (Eds.), Discursive Psychology and Embodiment: Beyond Subject-Object Binaries (pp. 81–111). Cham: Palgrave Macmillan. https://doi.org/http://link.springer.com/10.1007/978-3-030-53709-8_4
Dennett, D. C. (1987). The Intentional Stance. Cambridge, MA: MIT Press.
Due, B. L. (2021a). RoboDoc: Semiotic resources for achieving face-to-screenface formation with a telepresence robot. Semiotica, 2021(238), 253–278. https://doi.org/10.1515/sem-2018-0148
Due, B. L. (2021b). Distributed Perception: Co‐Operation between Sense‐Able, Actionable, and Accountable Semiotic Agents. Symbolic Interaction, 44(1), 134–162. https://doi.org/10.1002/symb.538
Edwards, D. (1997). Discourse and Cognition. London: Sage Publications. Retrieved from http://www.uk.sagepub.com/books/9780803976979
Enfield, N. J. (2013). Relationship Thinking: Agency, Enchrony, and Human Sociality. USA: OUP.
Enfield, N. J. (2014). Human agency and the infrastructure for requests. In P. Drew & E. Couper-Kuhlen (Eds.), Requesting in Social Interaction (pp. 35–54). Amsterdam: John Benjamins Publishing Company.
Fischer, K. (2011). Interpersonal variation in understanding robots as social actors. In 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 53–60). https://doi.org/10.1145/1957656.1957672
Fischer, K. (2021). Tracking Anthropomorphizing Behavior in Human-Robot Interaction. ACM Transactions on Human-Robot Interaction, 11(1). https://doi.org/10.1145/3442677
Forlizzi, J., & DiSalvo, C. (2006). Service Robots in the Domestic Environment: A Study of the Roomba Vacuum in the Home. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (pp. 258–265). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/1121241.1121286
Garfinkel, H. (1967). Studies in Ethnomethodology. Englewood Cliffs NJ: Prentice-Hall.
Garfinkel, H. (1991). Respecification: evidence for locally produced, naturally accountable phenomena of order*, logic, reason, meaning, method, etc. in and as of the essential haecceity of immortal ordinary society (I): an announcement of studies. In G. Button (Ed.), Ethnomethodology and the Human Sciences (pp. 10–19). Cambridge, U.K.: Cambridge University Press.
Goffman, E. (1981). Forms of Talk. Philadelphia: University of Pennsylvania Press.
Goodwin, C. (2004). A Competent Speaker Who Can't Speak: The Social Life of Aphasia. Journal of Linguistic Anthropology, 14(2), 151-170. https://doi.org/https://doi.org/10.1525/jlin.2004.14.2.151
Goodwin, C. (2018). Co-Operative Action. Cambridge: Cambridge University Press.
Greenbaum, J., & Kyng, M. (Eds.). (1991). Design at Work: Cooperative Design of Computer Systems. CRC Press.
Harjunpää, K. (2021). Brokering co-participants’ volition in request and offer sequences. In A. P. Jan Lindström Ritva Laury & M.-L. Sorjonen (Eds.), Intersubjectivity in Action: Studies in language and social interaction (pp. 135–159). Amsterdam: John Benjamins. Retrieved from https://www.jbe-platform.com/content/books/9789027259035-pbns.326.07har
Hedayati, H., Szafir, D., & Andrist, S. (2019). Recognizing F-Formations in the Open World. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 558–559). https://doi.org/10.1109/HRI.2019.8673233
Heritage, J. (1984). Garfinkel and Ethnomethodology. Cambridge [Cambridgeshire]; New York, N.Y: Polity Press.
Heritage, J., & Raymond, G. (2012). Navigating epistemic landscapes: acquiesence, agency and resistance in responses to polar questions. In J. P. de Ruiter (Ed.), Questions: Formal, Functional and Interactional Perspectives (pp. 179–192). Cambridge, U.K.: Cambridge University Press. https://doi.org/10.1017/CBO9781139045414.013
Hirschauer, S. (1991). The manufacture of bodies in surgery. Social Studies of Science, 21(2), 279–319.
Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition. ACM Transactions on Computer-Human Interaction, 7(2), 174–196. https://doi.org/10.1145/353485.353487
Hollnagel, E., & Woods, D. (2005). Joint Cognitive Systems: Foundations of Cognitive Systems Engineering. Boca Raton, FL: CRC Press.
International Organization for Standardization. (2021). Robotics — Vocabulary (ISO Standard No. 8373). Retrieved from https://www.iso.org/obp/ui/#iso:std:iso:8373:ed-3:v1:en
Jakonen, T., & Jauni, H. (2021). Mediated learning materials: visibility checks in telepresence robot mediated classroom interaction. Classroom Discourse, 12(1–2), 121–145. https://doi.org/10.1080/19463014.2020.1808496
Jung, M. F. (2017). Affective grounding in human-robot interaction. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (pp. 263–273). New York, NY, USA: ACM. https://doi.org/10.1145/2909824.3020224
Keevallik, L. (2017). Negotiating deontic rights in second position. In L. R. M.-L. Sorjonen & E. Couper-Kuhlen (Eds.), Imperative Turns at Talk: The Design of Directives in Action (pp. 271–295). Amsterdam: John Benjamins. https://doi.org/10.1075/slsi.30.09kee
Klowait, N., & Erofeeva, М. A. (2021). The Rise of Interactional Multimodality in Human-Computer Interaction. The Monitoring of Public Opinion: Economic & Social Changes, 1(1), 46–70. https://doi.org/10.14515/monitoring.2021.1.1793
Krummheuer, A. (2015a). Technical agency in practice: The enactment of artefacts as conversation partners, actants and opponents. PsychNology Journal, 13(2–3), 179–202.
Krummheuer, A. (2015b). Performing an action one cannot do: Participation, scaffolding and embodied interaction. Journal of Interactional Research in Communication Disorders, 6(2), 187–210. https://doi.org/http://dx.doi.org/10.1558/jircd.v6i2.26986
Laurier, E., Maze, R., & Lundin, J. (2006). Putting the Dog Back in the Park: Animal and Human Mind-in-Action. Mind, Culture, and Activity, 13(1), 2–24. https://doi.org/10.1207/s15327884mca1301_2
Latour, B., 2005. Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford UP.
Licoppe, C. (2021). The politics of visuality and talk in French courtroom proceedings with video links and remote participants. Journal of Pragmatics, 178, 363–377. https://doi.org/10.1016/j.pragma.2021.03.023
Linell, P. (2009). Rethinking Language, Mind, and World Dialogically. Charlotte, NC: Information Age Publishing.
Lipp, B. (2019). Interfacing RobotCare – On the Techno-Politics of Innovation. Technische Universität München, Munich, Germany. Retrieved from https://mediatum.ub.tum.de/doc/1472757/1472757.pdf
Miksell, L. (2010). Repetitional responses in frontotemporal dementia discourse: Asserting agency or demonstrating confusion? Discourse Studies, 12(4), 465–500. https://doi.org/10.1177/1461445610370127
Mondada, L. (2019). Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics, 145, 47–62. https://doi.org/10.1016/j.pragma.2019.01.016
Nass, C., & Moon, Y. (2000). Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues, 56(1), 81–103. https://doi.org/10.1111/0022-4537.00153
Otsuka, K., Yamato, J., Takemae, Y., & Murase, H. (2006). Conversation Scene Analysis with Dynamic Bayesian Network Basedon Visual Head Tracking. In 2006 IEEE International Conference on Multimedia and Expo (pp. 949–952). IEEE. https://doi.org/10.1109/ICME.2006.262677
Parenti, L., Marchesi, S., Belkaid, M., & Wykowska, A. (2021). Exposure to Robotic Virtual Agent Affects Adoption of Intentional Stance. In Proceedings of the 9th International Conference on Human-Agent Interaction (pp. 348–353). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3472307.3484667
Pelikan, H. R. M., Broth, M., & Keevallik, L. (2020). “Are You Sad, Cozmo?”: How Humans Make Sense of a Home Robot’s Emotion Displays. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 461–470). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3319502.3374814
Peräkylä, A. (2002). Agency and authority: extended responses to diagnostic statements in primary care encounters. Research on Language and Social Interaction, 35(2), 219–247. https://doi.org/10.1207/S15327973RLSI3502_5
Pitsch, K. (2016). Limits and Opportunities for Mathematizing Communicational Conduct for Social Robotics in the Real World? Toward Enabling a Robot to Make Use of the Human’s Competences. AI & Society, 31(4), 587–593. https://doi.org/10.1007/s00146-015-0629-0
Porcheron, M., Fischer, J. E., & Reeves, S. (2020). Pulling back the curtain on the Wizards of Oz. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3). https://doi.org/10.1145/3432942
Reeves, B., & Nass, C. (1996). The Media Equation. How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge: Cambridge University Press.
Sacks, H. (1992). Lectures on conversation. (G. Jefferson, Ed.). Oxford: Blackwell.
Schegloff, E. A. (2007). Sequence Organization in Interaction. Sequence Organization in Interaction: A Primer in Conversation Analysis I. Cambridge: Cambridge University Press.
Schlosser, M. (2019). Agency. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Winter 2019 Edition). Metaphysics Research Lab, Stanford University. Retrieved from https://plato.stanford.edu/archives/win2019/entries/agency/
Sheridan, T. B., & Verplank, W. L. (1978). Human and Computer Control of Undersea Teleoperators. Cambridge, MA. Retrieved from https://apps.dtic.mil/sti/citations/ADA057655
Skantze, G. (2021). Turn-taking in Conversational Systems and Human-Robot Interaction: A Review. Computer Speech & Language, 67, 101178. https://doi.org/https://doi.org/10.1016/j.csl.2020.101178
Sterponi, L. (2003). Account episodes in family discourse: the making of morality in everyday interaction. Discourse Studies, 5(1), 79–100. https://doi.org/10.1177/14614456030050010401
Suchman, L. A. (1987). Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge: Cambridge University Press.
Suchman, L. A. (2007). Human-machine reconfigurations: Plans and situated actions, 2nd ed. New York, NY, US: Cambridge University Press.
Thellman, S., & Ziemke, T. (2021). The Perceptual Belief Problem: Why Explainability Is a Tough Challenge in Social Robotics. J. Hum.-Robot Interact., 10(3).
Traum, D. R. (1994). A Computational Theory of Grounding in Natural Language Conversation. Rochester University, Rochester, NY, USA. Retrieved from https://apps.dtic.mil/sti/citations/ADA289894
Waring, H. Z. (2011). Learner initiatives and learning opportunities. Classroom Discourse, 2(2), 201–218. https://doi.org/10.1080/19463014.2011.614053
Warnicke, C., & Broth, M. (forthcoming). Making Two Actions at Once: How interpreters address different parties simultaneously in the Swedish Video Relay Service. Translation and Interpreting Studies.
Weatherall, A. (2020). Constituting Agency in the Delivery of Telephone-Mediated Victim Support. Qualitative Research in Psychology, 17(3), 396–412. https://doi.org/10.1080/14780887.2020.1725951
Weatherall, A., & Keevallik, L. (2016). When claims of understanding are less than affiliative. Research on Language and Social Interaction, 49(3), 167–182. https://doi.org/10.1080/08351813.2016.1196544
Wilson, G., & Shpall, S. (2012). Action. In Edward N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Winter 2016 Edition). Retrieved from https://plato.stanford.edu/archives/win2016/entries/action
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Author and Journal
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
We follow the Budapest Open Access Initiative's definition of Open Access.
The journal allows the author(s) to hold the copyright without restrictions.
The journal allows software/spiders to automatically crawl the journal content (also known as text mining)
The journal provides article level metadata to DOAJ
The journal allows readers to read, download, distribute, print, search, or link to the full texts of its articles and allow readers to use them for any other lawful purpose.
