Web versus papir: En sammenligning af kvalitet i surveydata
Nøgleord:
Surveyforskning, metode, indsamlingsmetode, svarprocenter, web-baseret, satisficing, web mode effekt, skævhedResumé
Surveyforskningen står overfor en række udfordringer i form af eksempelvis faldende svarprocenter og øget skævhed i data. Dette har medført en stigende afprøvning af nye indsamlingsformer, hvor særligt de web-baserede surveys har vundet frem. I den videnskabelige litteratur er der dog en vis skepsis overfor dataindsamling online, da der argumenteres for, at kvaliteten af datamaterialet forringes, primært fordi respondenten på nettet ikke gør sig samme kognitive indsats for at besvare spørgsmålene, og derfor er mere tilbøjelig til ”satisficing”, end hvis de havde besvaret skemaet eksempelvis i et telefoninterview.
Denne artikel stiller imidlertid spørgsmålstegn ved, om eventuelle forskelle i datakvalitet på baggrund af indsamlingsmetode kan tilskrives brugen af nettet per se, da andre faktorer, som eksempelvis tilstedeværelsen af en interviewer i processen, ligeledes menes at kunne influere på de svar, som respondenterne giver. Vi har derfor foretaget et lille eksperimentelt studie, hvor eneste varierende faktor mellem to tilfældigt udvalgte grupper af studerende er brugen af indsamlingsmetoden. Resultaterne viser, at der ikke kan genfindes den såkaldte ”web mode effekt”, idet intet tyder på, at de respondenter, der har udfyldt den online udgave af spørgeskemaet, har større tendens til ”satisficing” end de respondenter, der har udfyldt skemaet i papirform.
Referencer
Bethlehem, Jelke & Silvia Biffignandi (2012): Handbook of Web Surveys. Hoboken: Wiley
Biemer, Paul P. & Lars Lyberg (2003) Introduction to survey quality. Hoboken: Wiley
Blasius, Jörg & Victor Thiessen (2012): Assessing the Quality of Survey Data. London: SAGE
Bradley, Nigel (1999) “Sampling for internet surveys: an examination of respondent selection for internet research”. In Journal of Market Research Society, Vol. 41, No. 4, pp. 387-394
Braunsberger, Karin, Hans Wybenga & Roger Gates (2007) “A comparison of reliability between telephone and web-based surveys”. In Journal of Business Research 60 (2007) 758-764
Chang, Linchiat & Jon A. Krosnick (2010): “Comparing oral interviewing with self-administered computer-ized questionnaires. An experiment”, in Public Opinion Quarterly, Vol. 74, No. 1, pp. 154-167
Chang, Linchiat & Jon A. Krosnick (2009): “National Surveys via RDD Telephone Interviewing versus the Internet. Comparing sample representativeness and response quality”, in Public Opinion Quarterly, Vol. 73, No. 4, pp. 1-38
Couper, Mick P. (2011): “The future of modes of data collection”, in Public Opinion Quarterly, Vol. 75, No. 5, pp. 889-908
Couper, Mick P. (2000) “Web surveys. A review of issues and approaches”. In Public Opinion Quarterly, vol. 64, pp. 464-494
Crawford, Scott D., Mick P. Couper & Mark J. Lamias. (2001). “Web surveys. Perceptions of Burden”. In Social Science Computer Review Vol. 19, No 2, pp. 146-62.
Dalton, Russel J., Scott Flanagan & Paul Allen Beck (1984) Electoral change in advanced industrial democracies Realignment or dealignment? Princeton, NJ: Princeton University Press
De Leeuw, Edith D. (2008): “Choosing the method of data collection”, in E.D. de Leeuw, J.J. Hox & D.A. Dillman (eds.): International Handbook of Survey Methodology, pp. 113-135. New York: Psychology Press
de Leeuw, Edith D. (2005) “To mix or not to mix Data collection modes in surveys”. In Journal of Official Statistics, Vol. 21, No. 2, pp. 233-255
de Leeuw, Edith D. & Walter de Heer (2002) “Trends in Household Survey Nonresponse: A Longitudinal and International Comparison” In Robert M. Groves, Don A. Dillman, John L. Eltinge & Roderick J. A. Little (Eds.) Survey Nonresponse. New York: Wiley
de Leeuw, Edith D. & Johannes Van Der Zowen (1988) “Data quality in telephone and face-to-face surveys: A comparative analysis”. In Robert M. Groves, Paul P. Biemer, Lars E. Lyberg, James T. Massey, William L. Nicholls II & Joseph Waksberg (Eds.) Telephone Survey Methodology. New York: Wiley
de Vaus, David (2002) Surveys in social research. London: Routledge
Dillman, Don A (2000) Mail and Internet Surveys: The Tailored Design Method (2nd ed.). New York: Wiley
Dillman, Don A., Jolene D. Smyth & Leah Melani Christian (2009) Internet, mail and mixed-mode surveys. The tailored design method. New York: Wiley.
Fricker, Scott, Mirta Galestic, Roger Tourangeu & Ting Yan (2005) “An experimental comparison of Web and telephone survey”. In Public Opinion Quarterly, Vol. 69, No. 3, pp. 370-392
Frippiat, Didier, Nicolas Marquis & Elizabeth Wiles-Portier (2010): “Web Surveys in the Social Sciences: An Overview”, in Population, Vol. 65, No. 2, pp. 285-311.
Greene, Jessica, Howard Speizer & Wyndy Wiitala (2008) “Telephone and Web: Mixed-Mode Challenge”. In Health Services Research, Vol. 43, No, 1, pp. 230-248
Heerwegh, Dirk & Geert Loosveldt (2008) “Face-to-face versus web surveying in a high-internet-coverage population. Differences in response quality”. In Public Opinion Quarterly, Vol. 72, No.5 2008, pp. 836-846
Hoek Janet, Phillip Gendall & Benjamin Healey (2002) “Web-based polling: An evaluation of survey modes”. In Australasian Journal of Market Research, Vol.10, No. 2, pp. 25-35
Holbrook, Allyson, Melaine C. Green & Jon A. Krosnick (2003) “Telephone versus face-to-face interviewing of national probability samples with long questionnaires. Comparisons of respondent satisficing and social desirability response bias”. In Public Opinion Quarterly Volume 67:79-125
Hox, Joop J. & Edith D de Leeuw (1994). “A Comparison of Nonresponse in Mail, Telephone, and Face to face Surveys: Applying Multilevel Modeling to Meta-analysis”. In Quality and Quantity, Vol. 28, No. 4, pp. 329-344
Jäckle, Anette, Caroline Roberts & Peter Lynn (2010): “Assessing the Effekts of Data Collection Mode on Measurement”, in International Statistical Review, 78, 1, pp. 3-20
Jäckle, Annette, Caroline Roberts, & Peter Lynn (2006). Telephone versus face-to-face interviewing: Mode effekts on data quality and likely causes (Report on phase II of the ESS-Gallup Mixed Mode Methodology Project). Colchester: University of Essex
Kaye, Barbara K. & Thomas J. Johnson (1999) “Methodology: Taming the Cyber Frontier. Techniques for Improving Online Surveys.” In Social Science Computer Review, Vol. 17, No. 3, pp. 323-337
Klausch, Thomas, Joop J. Hox & Barry Schouten (2013): “Measurement Effekts of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions”, in Sociological Methods & Research, 42(3), pp. 227-263
Krosnick, Jon A. (1991) ”Response strategies for coping with the cognitive demands of attitude measures in surveys”. In Applied Cognitive Psychology, Vol. 5, pp. 2113-236
Krosnick, Jon A., Sowmya Narayan & Wendy R. Smith (1996) “Satisficing in surveys. Initial evidence” in Marc T. Braveman & Jana Kay Slater (Eds.) Advances in survey research. San Francisco: Jossey-Bass.
Petty, Richard E. & John T. Cacioppo (1982) “The Need for Cognition” In Journal of Personality and Social Psychology, Vol. 42, No. 1, 116-131
Revilla, Melanie A. & Willem E. Saris (2013): “A Comparison of the Quality of Questions in a Face-to-face and a Web Survey”, in International Journal of Public Opinion Research, 2012, pp. 1-12
Roberts, Caroline E. (2007) “Mixing modes of data collection in surveys: A methodological review”. ESRC National Centre for Research Methods, NCRM Methods Review Papers, NCRM/008
Roberts, Caroline, Annette Jäckle & Peter Lynn (2007) “Causes of mode effekts: Separating out interviewer and stimulus effekts in comparisons of face-to-face and telephone surveys”. Proceedings of the Social Research Methods Section of the American Statistical Association
Roster, Catherine A., Robert D. Rogers, Gerald Albaum & Darin Klein (2004) “A comparison of response characteristics from Web and telephone surveys”. In International Journal of Market Research, Vol.46 No. 3, pp. 359-373
Sax, Linda J., Shannon K. Gilmartin & Alyssa N. Bryant (2003) “Assessing response rates and nonresponse bias in Web and paper surveys”. In Research in Higher Education, Vol. 44, No. 4, pp. 409-432
Smith, Tom W. (2013): “Survey-Research Paradigms Old and New”, in International Journal of Public Opin-ion Research, Vol. 3, No. 2, pp. 218-229
Solomon, David J. (2001) “Conducting Web-Based Surveys”. In Practical Assessment, Research & Evaluation, Vol. 7, No. 19
Tourangeau, Roger, Lance J. Rips & Kenneth Rasinski (2000) The Psychology of Survey Response. Cambridge: Cambridge University Press
