Last time, I discussed John Cacioppo’s lecture at MSU on the direction of social psychology. In addition to increasing collaboration within (and across) disciplines, he also highlighted the importance of increasing the public knowledge and credibility of psychology as a discipline.
Although John wasn’t specifically referring to instances of scandals across the discipline, the truth is that there have been many of these troubling instances in recent years. In 2011 alone, Daryl Bem’s paper on ESP appeared in social psychology’s flagship journal (Journal of Personality and Social Psychology) suggesting that in certain cases, humans have the ability to “feel the future.” In addition, noted social psychologist Deiderik Stapel was found to have fabricated data for over ten years of research, a story which made headlines in major news outlets such as the New York Times.
Putting Bem’s research momentarily aside (which has been critiqued in many other blogs, conferences, and news articles), the issue that most troubles me is how Stapel was able keep his fabrication a secret for so long, despite his work often appearing in news headlines, both in major journals (i.e., Science) and major news outlets. According to the NYT:
“He was “lord of the data,” the only person who saw the experimental evidence that had been gathered (or fabricated). This is a widespread problem in psychology, said Jelte M. Wicherts, a psychologist at the University of Amsterdam. In a recent survey, two-thirds of Dutch research psychologists said they did not make their raw data available for other researchers to see. “This is in violation of ethical rules established in the field,” Dr. Wicherts said.”
Ethics aside, the easiest way to prevent another incident like Deiderik Stapel is to promote—mandate even—the sharing of published data. Indeed, APA guidelines (8.14) explicitly says as much, “After research results are published, psychologists do not withhold the data on which their conclusions are based from other competent professionals who seek to verify the substantive claims through reanalysis.”
However, despite this guideline, many researchers don’t share their data. Why is this? First, let’s assume that it is not because of data fabrication. There are many reasons to withhold data from a published paper, fabrication being (hopefully) only a very small fraction of the actual reasons. I list a few of the common reasons.
1) Difficulty of sharing. Back before data was routinely captured and saved digitally, it was often not a simple matter to share data with a curious researcher. APA guidelines still reflect this difficulty, maintaining that requests for sharing of data do not “preclude psychologists from requiring that such individuals or groups be responsible for costs associated with the provision of such information.” However, with the advent of digital storage and data analysis, years of data is often at a researcher’s fingertips, and can be easily sent to other researchers via email. That the data is unorganized is not a legitimate excuse for refusing the request, although it would be well within the researcher’s rights to request a period of time to assemble the data.
2) Confidentiality of participants. Of course, when the data can be connected to specific individuals, as in the case of video recordings or data that has personal identifiers, such data may be restricted from sharing to protect the confidentiality of the participants. While protecting confidentiality is of the utmost importance, the fact remains that most of the research conducted in social psychology is disassociated from individuals, and so does not fall under this restriction.
3) Data “theft.” Some psychologists may be reluctant to give out data for fear that other researchers will obtain their data and publish papers based on that data without sharing the credit. While this is a serious concern, it is well within the bounds of the original researcher to have the requester promise to credit him or her if the requester analyzes the data differently and wishes to publish it. But it makes no sense for the original researcher to refuse to share data on the premise that the requester wants to double check the analyses that were used to support the paper’s experimental hypotheses.
From the list of reasons above, excepting the small fraction of cases where data sharing would violate the confidentiality of participants, the arguments against sharing data are impoverished and out of date. Additionally, in the case of data theft, if any time a study was published the data were to be published alongside the written report, it would be clear if another researcher used the same set of data and did not credit the original author when his or her data and report were published.
For this reason, I propose (and I am not the first), that data should be transparent, and that it should be a basic requirement of major journals (e.g., Journal of Personality and Social Psychology; Psychological Science) that data be released with the publication of a paper. Why not let other psychologists engage in peer review? In this way we can get to the bottom of fabricated data, or expose critical flaws in methodology that otherwise lead to off the wall findings.
Like Cacioppo warned, for psychology to survive, we have to maintain public awareness to both the utility and credibility of psychology. I see transparency of data as an easy first step in this direction that could be implemented immediately. Aside from minor website changes to accompany the presence of data with a written report, there is really no reason why data could not be “published” with the written report. And yet this is only one step in the right direction of salvaging and fostering credibility within our field. Although some researchers are optimistic about the future, without serious changes in how we report data, more incidents like these will continue to deteriorate confidence in our work and our discipline as a whole. There’s a storm coming, and if we cannot—or will not—see it, we will undermine the very work we are doing.