Replication

Conceptions of Replication

Replication is typically broken into two primary categories: direct and conceptual. Direct replications follow the original study’s methods (e.g., similar participants, measures, and procedures) as closely as possible in order to test whether the original research results can be obtained again. Conceptual replications purposefully alter at least one component of the original study in order to test the underlying construct of interest. Replications serve as a critical part of the scientific enterprise by helping control for issues caused by sampling error, artifacts, fraud, generalizability, [Page 1410]testing the general underlying hypothesis of a previous study, or some combination of those issues.

Conceptual replications can help identify the degree to which a particular finding is broad or narrow. A reanalysis of previously collected data is not a replication but is still a vital part of the scientific process. Reanalysis of data can reveal things ranging from simple calculation errors to more fundamental problems such as data manipulation and fraud, and reanalysis can provide new information by reanalyzing seminal data sets using modern statistical techniques.

There is not universal agreement about the necessary and sufficient features of a replication, despite its being one of the basic building blocks of science. A 2009 review by Stefan Schmidt connects replication theory with replication in practice. Schmidt lists five-function replications that serve to control for sampling error, to control for artifacts, to control for fraud, to generalize to different or larger populations, or to assess the general hypothesis of a previous study. Rather than deliberately avoiding the original methods, Schmidt suggests systematically changing individual facets of the original study to better understand its nature.

The relative importance of direct and conceptual replications has been debated. Some scholars argue that conceptual replication should be emphasized, whereas others support direct replications. The importance of each depends on the goal of the investigation, with direct replication typically seeking to verify or corroborate the original findings using the same methods as the original researchers; conceptual replications test more general models and theories. However, it is important to note that only direct replications can disconfirm or corroborate previous claims. This is because a failed conceptual replication does not automatically identify a flaw in the original study but instead has the potential to identify the generalizability (or lack thereof) of the original finding. Direct replication can help identify potential biases in the original study or confirm that the original finding was not an anomaly. Because of this, some scholars argue that direct replication should always precede conceptual replication attempts.

...