File Drawer Problem

The file drawer is a metaphorical term referring to a storage location for nonpublished research. The file drawer problem, a term coined by Robert Rosenthal, refers to the possibility that nonpublished results differ systematically from published results. Systematic differences between published and nonpublished research are especially problematic for the field of education, where summaries of research through meta-analysis are increasingly relied upon to inform practice. This entry describes the nature, causes, and consequences of the file drawer problem as well as the methods for its detection and eradication.

Of all studies conducted by researchers, some become published and easily accessible to consumers. Other studies are said to be relegated to the file drawer. The file drawer problem is one type of publication bias, a broader phenomenon whereby published research is a nonrepresentative sample of all research. Reasons for a research manuscript not being accepted for publication are often linked to reviewer or editorial bias against null or nonsignificant results during the peer review process.

Consequences of publication bias became salient with the rise of meta-analysis. Indeed, meta-analytic inferences rest on the assumption that the included studies constitute an unbiased sample. In the modal case, the concern is with upward bias; that is, some studies with small or null effects are missing from the summary, resulting in unrealistically high meta-analytic estimates. As a second consequence, an unrepresentative sample of published research can provide unrealistically low estimates of the reproducibility of scientific research. Indeed, if only the “best-looking” findings are selected for publication, then replication attempts are increasingly likely to fail. Finally, publication bias has the potential to stymie attempts at evidence-based practice. Indeed, failures of evidence application should increase with the level of bias associated with the evidence.

Meta-analysts have developed several techniques for detecting and correcting for the impact of publication bias. Indeed, Rosenthal’s seminal approach provides an estimate of the number of file drawer studies with null results that would need to exist in order to affect one’s meta-analytic conclusions. Newer approaches provide revised meta-analytic estimates after imputing studies assumed to be contained in the file drawer or by making other modifications to the distribution of effects.

To completely eradicate publication bias would be preferable to improving its detection or assessment. Given that one culprit for the existence of publication bias is found in the journal editorial process, several journals have adopted modified peer review processes, wherein authors first [Page 678]submit manuscripts without findings and conclusions. Then, after peer review for rigor and relevance has been completed, the results and discussion sections are submitted. As another culprit for publication bias, authors might simply abandon research projects without having submitted them for publication. There are now several mechanisms available to reduce this concern, such as those provided by the Center for Open Science, that allow researchers to upload and make available research data, manuscripts, and the like.

See also Effect Size; Meta-Analysis; Missing Data Analysis; Quantitative Research Methods; Threats to Research Validity; Type I Error