If youâ€™re a psychologist, the news has to make you a little nervousâ€”particularly if youâ€™re a psychologist who published an article in 2008 in any of these three journals:Psychological Science, the Journal of Personality and Social Psychology,or the Journal of Experimental Psychology: Learning, Memory, and Cognition.
Because, if you did, someone is going to check your work. A group of researchers have already begun what theyâ€™ve dubbed the Reproducibility Project, which aims to replicate every study from those three journals for that one year. The project is part of Open Science Framework, a group interested in scientific values, and its stated mission is to â€œestimate the reproducibility of a sample of studies from the scientific literature.â€ This is a more polite way of saying â€œWe want to see how much of what gets published turns out to be bunk.â€
For decades, literally, there has been talk about whether what makes it into the pages of psychology journalsâ€”or the journals of other disciplines, for that matterâ€”is actually, you know, true. Researchers anxious for novel, significant, career-making findings have an incentive to publish their successes while neglecting to mention their failures. Itâ€™s what the psychologist Robert Rosenthal named â€œthe file drawer effect.â€ So if an experiment is run ten times but pans out only once you trumpet the exception rather than the rule. Or perhaps a researcher is unconsciously biasing a study somehow. Or maybe he or she is flat-out faking results, which is not unheard of. Diederik Stapel, weâ€™re looking at you.