Pilot Studies

A pilot study is a research study that tests the feasibility of an approach that will later be used in a larger study. Pilot studies are conducted in quantitative and qualitative research. They can be extremely useful in providing justification or testing procedures for a larger future study. Pilot studies can benefit researchers by providing a “dress rehearsal” that saves time and avoids problems in the later study.

Pilot studies are not intended to test the hypothesis or research question for the larger study. Rather, pilot studies are meant to assess feasibility. Assessing the pragmatics of recruitment efforts, research instruments, randomization and data collection procedures, training sessions for staff, collaborative efforts, and intervention implementations are reasons why a pilot study is conducted. For example, a researcher may have concerns as to whether students would consent to being videoed during an actual class examination—a requirement of participants who enroll in a larger study investigating test anxiety. Because the researchers are worried about recruitment efforts, they may complete a pilot study initially to assess these efforts. Or the researcher may want to determine whether a 10-item multiple choice instrument is consistently read and understood by students who take it. In this instance, the researcher may opt to complete a pilot study to assess how the measurement instrument performs prior to using it in a larger study.

Because pilot studies are not focused on hypothesis testing, the sample size for a pilot study is [Page 1254]often small. The sample size of the pilot study only needs to be large enough to provide meaningful information about the aspects that are being assessed for feasibility.

Completed studies with a sample size that did not meet the power analysis established a priori should not be referenced as a pilot study. This is a common error seen in the literature. Calling a study a pilot simply because it has a small sample size is incorrect. As mentioned previously, a pilot study focuses on feasibility, which will have different questions and objectives from the larger main study.

Some researchers use pilot testing as a means to determine effect sizes and sample size determination for the main study. This is controversial in the literature with some authors arguing that this process exceeds the limits of what a pilot study can do because of the small sample size. If pilot studies are used in this manner, it should be done cautiously, especially with treatment effects, as the estimates from the pilot study may be unrealistic or biased due to the small sample size.

See also Effect Size; Power Analysis; Qualitative Research Methods; Quantitative Research Methods