Eye-Tracking in Virtual Reality
A Visceral Notice Approach for Protecting Privacy
DOI:
https://doi.org/10.7146/psj.v2i.134656Keywords:
eye-tracking, privacy, visceral notice, virtual reality, metaverseAbstract
Eye-tracking is in our future. Across many fields, eye-tracking is growing in prominence. This paper focuses on eye-tracking in virtual reality as a case study to illuminate novel privacy risks and propose a governance response to them: a design shift that provides users with an experientially resonant means of understanding privacy threats. It is a strategy that Ryan Calo calls “visceral notice.” To make our case for visceral notice, we proceed as follows. First, we provide a concise account of how eye-tracking works, emphasizing its threat to autonomy and privacy. Second, we discuss the sensitive personal information that eye-tracking reveals, complications that limit what eye-tracking studies establish, and the comparative advantage large technology companies may have when tracking our eyes. Third, we explain why eye-tracking will likely be crucial for developing virtual reality technology. Fourth, we review Calo’s conception of visceral notice and offer suggestions for applying it to virtual reality to help users better appreciate eye-tracking risks. Finally, we consider seven objections to our proposals and provide counterpoints to them.
References
Ahlström, Christer, Katja Kircher, Marcus Nyström, and Benjamin Wolfe. ‘Eye Tracking in Driver Attention Research—How Gaze Data Interpretations Influence What We Learn’. Frontiers in Neuroergonomics 2 (2021). https://www.frontiersin.org/articles/10.3389/fnrgo.2021.778043.
Arafat, Imtiaz Muhammad, Sharif Mohammad Ferdous, and John Quarles. ‘The Effects of Cybersickness on Persons with Multiple Sclerosis.’ In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, 51–59, 2016. https://doi.org/https://doi.org/10.1145/2993369.2993383.
Bailenson, Jeremy. ‘Protecting Nonverbal Data Tracked in Virtual Reality’. JAMA Pediatrics 172, no. 10 (2018): 905–6. https://doi.org/https://doi.org/10.1001/jamapediatrics.2018.1909.
Ball, Matthew. The Metaverse: And How It Will Revolutionize Everything. Liveright Publishing Corporation, 2022.
Bar-Zeev, Avi. ‘VR and AR Eye-Tracking Technology Will Usher in a Privacy Dystopia.’ Vice, 28 May 2019. https://www.vice.com/en/article/bj9ygv/the-eyes-are-the-prize-eye-tracking-technology-is-advertisings-holy-grail.
Bergstrom, Jennifer Romano, and Andrew Jonathan Schall. ‘Eye Tracking in User Experience Design.’ ScienceDirect, 2014. https://doi.org/https://doi.org/10.1016/C2012-0-06867-6.
Bhave, Devasheesh P., Laurel H. Teo, and Reeshad S. Dalal. ‘Privacy at Work: A Review and a Research Agenda for a Contested Terrain’. Journal of Management 46, no. 1 (2020): 127–64. https://doi.org/10.1177/0149206319878254.
Bye, Kent. State of Privacy in XR & Neuro-Tech: Conceptual Frames. YouTube. Presented at the VRARA Global Summi, 2021. https://www.youtube.com/watch?v=pIpD4-gYImU.
Cakebread, Caroline. ‘You’re Not Alone, No One Reads Terms of Service Agreements’. Insider, 2017. https://www.businessinsider.com/deloitte-study-91-percent-agree-terms-of-service-witho ut-reading-2017-11.
Calo, Ryan. ‘Against Notice Skepticism In Privacy (And Elsewhere).’ 87 Notre Dame Law Review 1027, 21 March 2011. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1790144.
Calo, Ryan. ‘Code, Nudge, or Notice?’ Iowa Law Review 99, no. 2 (2013): 773–802. https://doi.org/https://ssrn.com/abstract=2217013.
Chalmers, David J. ‘Reality +: Virtual Worlds and the Problems of Philosophy’, xiii–xiv. W.W. Norton & Company, New York, NY., 2022.
David-John, Brendan, Diane Hosfelt, Kevin Butler, and Eakta Jain. ‘A Privacy-Preserving Approach to Streaming Eye-Tracking Data’. IEEE Transactions on Visualization and Computer Graphics 27, no. 5 (2021): 2555–65. https://doi.org/10.1109/TVCG.2021.3067787.
David-John, Brendan, Sophie Jörg, Sanjeev Koppal, and Eakta Jain. ‘The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars’. IEEE Transactions on Visualization and Computer Graphics 26, no. 5 (May 2020): 1880–90. https://doi.org/10.1109/TVCG.2020.2973052.
David-John, Brendan, Ao Liu, and Lirong Xia. ‘Let It Snow: Adding Pixel Noise to Protect the User’s Identity.’, 2020. https://www.researchgate.net/publication/341830854_Let_It_Snow_Adding_pixel_ noise_to_protect_the_user’s_identity.
David-John, Brendan, Candace Peacock, Ting Zhang, T. Scott Murdison, Hrvoje Benko, and Tanya R. Jonker. ‘Towards Gaze-Based Prediction of the Intent to Interact in Virtual Reality.’, 2021. https://doi.org/https://dl.acm.org/doi/10.1145/3448018.3458008.
Dick, Ellysse. ‘Balancing User Privacy and Innovation in Augmented and Virtual Reality’. Information Technology and Innovation Foundation (ITIF), 4 March 2021. https://itif.org/publications/2021/03/04/balancing-user-privacy-and-innovation-augmented-and-virtual-reality/.
Eggers, Dave. The Every. London, United Kingdom: Penguin Books Ltd, 2021.
Elgarf, Maha, Slim Abdennadher, and Menna Elshahawy. ‘I-Interact: A Virtual Reality Serious Game for Eye Contact Improvement for Children with Social Impairment’. In Serious Games, edited by Mariano Alcañiz, Stefan Göbel, Minhua Ma, Manuel Fradinho Oliveira, Jannicke Baalsrud Hauge, and Tim Marsh, 10622:146–57. Cham: Springer International Publishing, 2017. https://doi.org/10.1007/978-3-319-70111-0_14.
Erbilek, M., M. Fairhurst, and M.C.D.C. Abreu. ‘Age Prediction from Iris Biometrics’. In 5th International Conference on Imaging for Crime Detection and Prevention (ICDP 2013), 1–5. London, UK: Institution of Engineering and Technology, 2013. https://doi.org/10.1049/ic.2013.0258.
Eskildsen, Anton Mølbjerg, and Dan Witzner Hansen. ‘Analysis of Iris Obfuscation: Generalising Eye Information Processes for Privacy Studies in Eye Tracking.’ In ACM Symposium on Eye Tracking Research and Applications, 1–10. ETRA ’21 Full Papers. New York, NY, USA: Association for Computing Machinery, 2021. https://doi.org/10.1145/3448017.3457385.
Frischmann, Brett M., and Evan Selinger. Re-Engineering Humanity. Cambridge.: Cambridge University Press, 2019.
Furht, Borko, ed. ‘Immersive Virtual Reality’. In Encyclopedia of Multimedia, 345–46. Boston, MA: Springer US, 2008. https://doi.org/10.1007/978-0-387-78414-4_85.
Goldinger, Stephen D., Yi He, and Megan H. Papesh. ‘Deficits in Cross-Race Face Learning: Insights from Eye Movements and Pupillometry.’ Journal of Experimental Psychology: Learning, Memory, and Cognition 35, no. 5 (2009): 1105–22. https://doi.org/10.1037/a0016548.
Haensel, Jennifer X., Tim J. Smith, and Atsushi Senju. ‘Cultural Differences in Mutual Gaze during Face-to-Face Interactions: A Dual Head-Mounted Eye-Tracking Study’. Visual Cognition 30, no. 1–2 (7 February 2022): 100–115. https://doi.org/10.1080/13506285.2021.1928354.
Hartzog, W. ‘Opinions ∙ The Case Against Idealising Control’. European Data Protection Law Review 4, no. 4 (2018): 423–32. https://doi.org/10.21552/edpl/2018/4/5.
Hayden, Scott. ‘Varjo’s Enthusiast Grade VR Headset Is Getting a Brain-Computer Interface’. Road to VR (blog), 31 May 2022. https://www.roadtovr.com/varjos-aero-open-brain-computer-bci/.
Heller, Brittan. ‘Reimagining Reality: Human Rights and Immersive Technology’. Harvard Kennedy School, 12 June 2020.
Heller, Brittan. ‘Watching Androids Dream of Electric Sheep: Immersive Technology, Biometric Psychography, and the Law’. Vanderbilt Journal of Entertainment & Technology Law 23, no. 1 (1 December 2020): 1. https://scholarship.law.vanderbilt.edu/jetlaw/vol23/iss1/1.
Heller, Brittan, and Avi Bar-Zeev. ‘The Problems with Immersive Advertising: In AR/VR, Nobody Knows You Are an Ad’. Journal of Online Trust and Safety 1, no. 1 (28 October 2021). https://doi.org/10.54501/jots.v1i1.21.
Hills, Peter J., and J. Michael Pake. ‘Eye-Tracking the Own-Race Bias in Face Recognition: Revealing the Perceptual and Socio-Cognitive Mechanisms’. Cognition 129, no. 3 (2013): 586–97. https://doi.org/10.1016/j.cognition.2013.08.012.
Hung, Shao-Chin, and Marisa Carrasco. ‘Feature-Based Attention Enables Robust, Long-Lasting Location Transfer in Human Perceptual Learning’. Scientific Reports 11, no. 1 (2021): 13914. https://doi.org/10.1038/s41598-021-93016-y.
Hutt, Stephen, Angela E.B. Stewart, Julie Gregg, Stephen Mattingly, and Sidney K. D’Mello. ‘Feasibility of Longitudinal Eye-Gaze Tracking in the Workplace’. Proceedings of the ACM on Human-Computer Interaction 6, no. ETRA (13 May 2022): 148:1-148:21. https://doi.org/10.1145/3530889.
Ion, Florence. ‘Meta’s Smartwatch Is Dead Before It Can Even Compete (Report)’. Gizmodo, 9 June 2022. https://gizmodo.com/meta-facebook-smartwatch-ar-glasses-portal-cancelled-de-1849041987.
Jaloza, Lisa Brown. ‘Inside Facebook Reality Labs: Wrist-Based Interaction for the next Computing Platform’. Tech at Meta, 18 March 2021. https://tech.fb.com/ar-vr/2021/03/inside-facebook-reality-labs-wrist-based-interaction-for-the-next-computing-platform/.
Jerome, Joseph, and Jeremy Greenberg. ‘Privacy & Autonomy Considerations in Emerging, Immersive Digital Worlds’. AUGMENTED REALITY + VIRTUAL REALITY, The Future of Privacy Forum, April 2021. https://fpf.org/wp-content/uploads/2021/04/FPF-ARVR-Report-4.16.21-Digital.pdf.
John, Brendan, Sanjeev Koppal, and Eakta Jain. ‘EyeVEIL: Degrading Iris Authentication in Eye Tracking Headsets’, 2019. https://jainlab.cise.ufl.edu/documents/2019_ETRA_pre_print_eyeVEIL.pdf.
Kaltheuner, Frederike. ‘An Interview with Arvind Narayanan’. In Fake AI. Manchester, UK: Meatspace Press, 2021. https://fakeaibook.com.
Kaminski, Margot. ‘Regulating Real-World Surveillance’. Washington Law Review, 1 January 2015. https://scholar.law.colorado.edu/faculty-articles/405.
Kardan, Omid, Laura Shneidman, Sheila Krogh-Jespersen, Suzanne Gaskins, Marc G. Berman, and Amanda Woodward. ‘Cultural and Developmental Influences on Overt Visual Attention to Videos’. Scientific Reports 7, no. 1 (2017): 11264. https://doi.org/10.1038/s41598-017-11570-w.
Katsini, Christina, Yasmeen Abdrabou, George E. Raptis, Mohamed Khamis, and Florian Alt. ‘The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions’. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–21. CHI ’20. New York, NY, USA: Association for Computing Machinery, 2020. https://doi.org/10.1145/3313831.3376840.
Khamis, Mohamed, Florian Alt, and Andreas Bulling. ‘The Past, Present, and Future of Gaze-Enabled Handheld Mobile Devices: Survey and Lessons Learned’. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, 1–17. MobileHCI ’18. New York, NY, USA: Association for Computing Machinery, 2018. https://doi.org/10.1145/3229434.3229452.
Kim, Nancy S. Consentability: Consent and Its Limits. Cambridge University Press, 2019.
Kim, Yeji. ‘Virtual Reality Data and Its Privacy Regulatory Challenges: A Call to Move Beyond Text-Based Informed Consent’. California Law Review, February 2022. https://www.californialawreview.org/print/virtual-reality-data-and-its-privacy-regulatory-challenges-a-call-to-move-beyond-text-based-informed-consent/.
Kröger, Jacob Leon, Otto Hans-Martin Lutz, and Florian Müller. ‘What Does Your Gaze Reveal About You? On the Privacy Implications of Eye Tracking’. In Privacy and Identity Management. Data for Better Living: AI and Privacy, edited by Michael Friedewald, Melek Önen, Eva Lievens, Stephan Krenn, and Samuel Fricker, 576:226–41. Cham: Springer International Publishing, 2020. https://doi.org/10.1007/978-3-030-42504-3_15.
Kröger, Jacob Leon, Otto Hans-Martin Lutz, and Philip Raschke. ‘Privacy Implications of Voice and Speech Analysis – Information Disclosure by Inference’. In Privacy and Identity Management. Data for Better Living: AI and Privacy, edited by Michael Friedewald, Melek Önen, Eva Lievens, Stephan Krenn, and Samuel Fricker, 576:242–58. Cham: Springer International Publishing, 2020. https://doi.org/10.1007/978-3-030-42504-3_16.
Kröger, Jacob Leon, Otto Hans-Martin Lutz, and Stefan Ullrich. ‘The Myth of Individual Control: Mapping the Limitations of Privacy Self-Management’. SSRN Electronic Journal, 2021. https://doi.org/10.2139/ssrn.3881776.
Kröger, Jacob Leon, Philip Raschke, and Towhidur Rahman Bhuiyan. ‘Privacy Implications of Accelerometer Data: A Review of Possible Inferences’. In Proceedings of the 3rd International Conference on Cryptography, Security and Privacy, 81–87. ICCSP ’19. New York, NY, USA: Association for Computing Machinery, 2019. https://doi.org/10.1145/3309074.3309076.
Lengyel, Gabor, Kevin Carlberg, Majed Samad, and Tanya Jonker. ‘Predicting Visual Attention Using the Hidden Structure in Eye-Gaze Dynamics’, 2021. https://tanyajonker.com/assets/publications/Lengyal_et_al_2021_EMICS.pdf.
Liu, Ao, Lirong Xia, Andrew Duchowski, Reynold Bailey, Kenneth Holmqvist, and Eakta Jain. ‘Differential Privacy for Eye-Tracking Data’. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, 1–10. ETRA ’19. New York, NY, USA: Association for Computing Machinery, 2019. https://doi.org/10.1145/3314111.3319823.
Madary, Michael, and Thomas K. Metzinger. ‘Real Virtuality: A Code of Ethical Conduct. Recommendations for Good Scientific Practice and the Consumers of VR-Technology’. Frontiers in Robotics and AI 3 (2016). https://www.frontiersin.org/articles/10.3389/frobt.2016.00003.
McAndrew, Francis T., and Sara S. Koehnke. ‘On the Nature of Creepiness’. New Ideas in Psychology 43 (1 December 2016): 10–15. https://doi.org/10.1016/j.newideapsych.2016.03.003.
McGill, Mark. ‘EXTENDED REALITY (XR) AND THE EROSION OF ANONYMITY AND PRIVACY.’ THE IEEE GLOBAL INITIATIVE ON ETHICS OF EXTENDED REALITY (XR) REPORT. The Institute of Electrical and Electronics Engineers Standards Association (IEEE SA), November 2021. https://standards.ieee.org/wp-content/uploads/import/governance/iccom/extended-reality- anonymity-privacy.pdf.
McLuhan, Marshall. In Understanding Media: The Extensions of Man: (6. Printing). New York, Toronto: McGraw-Hill, 1964.
Mikhailenko, Maria, Nadezhda Maksimenko, and Mikhail Kurushkin. ‘Eye-Tracking in Immersive Virtual Reality for Education: A Review of the Current Progress and Applications’. Frontiers in Education 7 (2022). https://www.frontiersin.org/articles/10.3389/feduc.2022.697032.
Moss, Felix Joseph, Roland Baddeley, and Nishan Canagarajah. ‘Eye Movements to Natural Images as a Function of Sex and Personality’. Edited by Luis M. Martinez. PLoS ONE 7, no. 11 (30 November 2012): e47870. https://doi.org/10.1371/journal.pone.0047870.
Mozilla Foundation. ‘New Anthology “Fake AI” Ruthlessly Debunks Ai Hype, Snake Oil, and Pseudoscience.’, 14 December 2021. https://foundation.mozilla.org/en/blog/new-anthology-fake-ai-ruthlessly-debunks-ai-hype -snake-oil-and-pseudoscience/.
Nair, Vivek, Gonzalo Munilla Garrido, and Dawn Song. ‘Exploring the Unprecedented Privacy Risks of the Metaverse’. arXiv, 26 July 2022. https://doi.org/10.48550/arXiv.2207.13176.
Nair, Vivek, Gonzalo Munilla Garrido, and Dawn Song. ‘Going Incognito in the Metaverse’. arXiv, 19 August 2022. https://doi.org/10.48550/arXiv.2208.05604.
Pahi, Suchismita, and Calli Schroeder. ‘Extended Privacy for Extended Reality: XR Technology Has 99 Problems and Privacy Is Several of Them’. SSRN Scholarly Paper. Rochester, NY, 28 August 2022. https://doi.org/10.2139/ssrn.4202913.
Pearlman, Kavya. ‘XRSI – XR Safety Initiative | Help Build Safe Immersive Environments’. XRSI – XR Safety Initiative, 20 December 2021. https://xrsi.org/.
Peck, Tabitha C., Laura E. Sockol, and Sarah M. Hancock. ‘Mind the Gap: The Underrepresentation of Female Participants and Authors in Virtual Reality Research’. IEEE Transactions on Visualization and Computer Graphics 26, no. 5 (2020): 1945–54. https://doi.org/10.1109/TVCG.2020.2973498.
Richards, Neil. Why Privacy Matters. New York, NY: Oxford University Press, 2022.
Richards, Neil M., and Woodrow Hartzog. ‘A Duty of Loyalty for Privacy Law’. SSRN Scholarly Paper. Rochester, NY, 3 July 2020. https://doi.org/10.2139/ssrn.3642217.
Richards, Neil M., and Woodrow Hartzog. ‘The Pathologies of Digital Consent’. SSRN Scholarly Paper. Rochester, NY, 11 April 2019. https://papers.ssrn.com/abstract=3370433.
Robertson, Adi. ‘The Meta Quest Pro Is a Cutting-Edge Headset Looking for an Audience’. The Verge, 11 October 2022. https://www.theverge.com/23393115/meta-quest-pro-vr-headset-hands-on-specs-price.
Selinger, Evan, and Woodrow Hartzog. ‘The Inconsentability of Facial Surveillance’. SSRN Scholarly Paper. Rochester, NY, 19 March 2020. https://papers.ssrn.com/abstract=3557508.
Selinger, Evan, and Kyle Whyte. ‘Is There a Right Way to Nudge? The Practice and Ethics of Choice Architecture: Practice and Ethics of Choice Architecture’. Sociology Compass 5, no. 10 (2011): 923–35. https://doi.org/10.1111/j.1751-9020.2011.00413.x.
Stark, Luke, and Jevan Hutson. ‘Physiognomic Artificial Intelligence’. Fordham Intellectual Property, Media and Entertainment Law Journal 32, no. 4 (1 January 2022): 922. https://ir.lawnet.fordham.edu/iplj/vol32/iss4/2.
Surden, Harry. ‘Structural Rights in Privacy’. SMU Law Review, 1 January 2007. https://scholar.law.colorado.edu/faculty-articles/346.
Susser, Daniel. ‘Notice After Notice-and-Consent: Why Privacy Disclosures Are Valuable Even If Consent Frameworks Aren’t’. SSRN Scholarly Paper. Rochester, NY, 1 January 2019. https://papers.ssrn.com/abstract=3346167.
Syrjämäki, Aleksi H., Poika Isokoski, Veikko Surakka, Tytti P. Pasanen, and Jari K. Hietanen. ‘Eye Contact in Virtual Reality – A Psychophysiological Study’. Computers in Human Behavior 112 (2020): 106454. https://doi.org/10.1016/j.chb.2020.106454.
Tobii. ‘Use Eye Tracking to Improve Training and Skills Assessment’, 2022. https://www.tobii.com/solutions/training-and-skills-assessment/.
Valliappan, Nachiappan, Na Dai, Ethan Steinberg, Junfeng He, Kantwon Rogers, Venky Ramachandran, Pingmei Xu, et al. ‘Accelerating Eye Movement Research via Accurate and Affordable Smartphone Eye Tracking’. Nature Communications 11, no. 1 (11 September 2020): 4553. https://doi.org/10.1038/s41467-020-18360-5.
Wenzlaff, Frederike, Peer Briken, and Arne Dekker. ‘Video-Based Eye Tracking in Sex Research: A Systematic Literature Review’. The Journal of Sex Research 53, no. 8 (12 October 2016): 1008–19. https://doi.org/10.1080/00224499.2015.1107524.
Wheeler, Tom. ‘If the Metaverse Is Left Unregulated, Companies Will Track Your Gaze and Emotions’. Time. Accessed 26 September 2022. https://time.com/6188956/metaverse-is-left-unregulated-companies-will-track-gaze-emotions/.
Williams, Niall L., and Tabitha C. Peck. ‘Estimation of Rotation Gain Thresholds Considering FOV, Gender, and Distractors’. IEEE Transactions on Visualization and Computer Graphics 25, no. 11 (2019): 3158–68. https://doi.org/10.1109/TVCG.2019.2932213.
Wu, Esther Xiu Wen, Bruno Laeng, and Svein Magnussen. ‘Through the Eyes of the Own-Race Bias: Eye-Tracking and Pupillometry during Face Recognition’. Social Neuroscience 7, no. 2 (2012): 202–16. https://doi.org/10.1080/17470919.2011.596946.
Yuan, M. L., G. G Chua, F. Farbiz, and S. Rahardja. ‘Eye Contact with a Virtual Character Using a Vision-Based Head Tracker’. In Computer and Information Sciences, edited by Erol Gelenbe, Ricardo Lent, Georgia Sakellari, Ahmet Sacan, Hakki Toroslu, and Adnan Yazici, 62:217–25. Dordrecht: Springer Netherlands, 2011. https://doi.org/10.1007/978-90-481-9794-1_43.
Zheng, Ting, Christoph H. Glock, and Eric H. Grosse. ‘Opportunities for Using Eye Tracking Technology in Manufacturing and Logistics: Systematic Literature Review and Research Agenda’. Computers & Industrial Engineering 171 (1 September 2022): 108444. https://doi.org/10.1016/j.cie.2022.108444.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Evan Selinger, Ely Altman, Shaun Foster
This work is licensed under a Creative Commons Attribution 4.0 International License.
