Perhaps Psychology’s Replication Crisis is a Theoretical Crisis that is Only Masquerading as a Statistical One
Keywords:Meehl, Psychology, Replication, Statistics, Theory
The “replication crisis” may well be the single most important challenge facing empirical psychological research today. It appears that highly trained scientists, often without understanding the potentially dire long-term implications, have been mishandling standard statistical procedures in the service of attaining statistical “significance.” Exacerbating the problem, most academic journals do not publish research that has not produced a “significant” result. This toxic combination has resulted in journals apparently publishing many Type I errors and declining to publish many true failures to reject H0. In response, there has been an urgent call from some psychologists that studies be registered in advance so that their rationales, hypotheses, variables, sample sizes, and statistical analyses are recorded in advance, leaving less room for post hoc manipulation. In this chapter, I argue that this “open science” approach, though laudable, will prove insufficient because the null hypothesis significance test (NHST) is a poor criterion for scientific truth, even when it is handled correctly. The root of the problem is that, whatever statistical problems psychology may have, the discipline never developed the theoretical maturity required. For decades it has been satisfied testing weak theories that predict, at best, only the direction of the effect, rather than the size of effect. Indeed, uncritical acceptance of NHST by the discipline may have served to stunt psychology’s theoretical growth by giving researchers a way of building a successful career without having to develop models that make precise predictions. Improving our statistical “hygiene” would be a good thing, to be sure, but it is unlikely to resolve psychology’s growing credibility problem until our theoretical practices mature considerably.
How to Cite
Copyright (c) 2021 Christopher D. Green
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
IRTP operates based on a non-exclusive publishing agreement, according to which the journal retains the right of first publication, but authors are free to subsequently publish their work. The copyright of all work rests with the author(s).
All content published in IRTP is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license (CC BY-NC-SA 4.0). This license allows authors and readers to share and adapt content for non-commercial purposes, provided that they abide by the following terms:
- Give credit to the original author(s)/creator(s) and attribution parties (i.e., IRTP);
- Provide a link to the original source, to the extent practicable;
- Include the copyright notice and/or indicate the corresponding Creative Commons license;
- Indicate what, if any, adaptations were made to the original; and
- Share adapted content under the same license as the original.