Perhaps Psychology’s Replication Crisis is a Theoretical Crisis that is Only Masquerading as a Statistical One

Authors

  • Christopher D. Green York University

DOI:

https://doi.org/10.7146/irtp.v1i2.127764

Keywords:

Meehl, Psychology, Replication, Statistics, Theory

Abstract

The “replication crisis” may well be the single most important challenge facing empirical psychological research today. It appears that highly trained scientists, often without understanding the potentially dire long-term implications, have been mishandling standard statistical procedures in the service of attaining statistical “significance.” Exacerbating the problem, most academic journals do not publish research that has not produced a “significant” result. This toxic combination has resulted in journals apparently publishing many Type I errors and declining to publish many true failures to reject H0. In response, there has been an urgent call from some psychologists that studies be registered in advance so that their rationales, hypotheses, variables, sample sizes, and statistical analyses are recorded in advance, leaving less room for post hoc manipulation. In this chapter, I argue that this “open science” approach, though laudable, will prove insufficient because the null hypothesis significance test (NHST) is a poor criterion for scientific truth, even when it is handled correctly. The root of the problem is that, whatever statistical problems psychology may have, the discipline never developed the theoretical maturity required. For decades it has been satisfied testing weak theories that predict, at best, only the direction of the effect, rather than the size of effect. Indeed, uncritical acceptance of NHST by the discipline may have served to stunt psychology’s theoretical growth by giving researchers a way of building a successful career without having to develop models that make precise predictions. Improving our statistical “hygiene” would be a good thing, to be sure, but it is unlikely to resolve psychology’s growing credibility problem until our theoretical practices mature considerably.

Author Biography

Christopher D. Green, York University

Christopher Green is a professor in the Historical, Theoretical, & Critical Studies of Psychology graduate program at York University in Toronto, Canada. He hold PhDs in cognitive psychology and in the philosophy of science, both from the University of Toronto. Much of his research has been on the history of psychology, including the 2019 book, Psychology and Its Cities (Routledge). He has pioneered digital methods of historical research, co-authoring more than a dozen articles that explore the development of the intellectual structure of psychology, using large databases of the journal literature through time. He is now brining digital methods to bear on the statistical and methodological origins and of the “replication crisis” in psychology. He has also written on the (lack of) theoretical unification in psychology, the aesthetics of the golden section, and the history of baseball. He has served as president of the Society for the History of Psychology, editor of the journal of the History of the Behavioral Sciences, and is the incoming editor of History of Psychology.

Downloads

Published

2021-12-13

How to Cite

Green, C. D. (2021). Perhaps Psychology’s Replication Crisis is a Theoretical Crisis that is Only Masquerading as a Statistical One. International Review of Theoretical Psychologies, 1(2). https://doi.org/10.7146/irtp.v1i2.127764