Pilot Study

A pilot study is a small-scale implementation of a larger study or of part of a larger study. Pilot studies last for shorter amounts of time and usually involve a smaller number of participants, sites, or organizations. Though traditionally associated with quantitative experimental design, pilot studies can be used in any methodological setting, especially when attempting to collect data in a new format or location or to simply examine potential roadblocks before full implementation.

A pilot study may also be viewed as a feasibility study. A feasibility study is completed to determine if the full study can be accomplished. Feasibility studies are practical when there is concern that a full-scale study may not be possible due to concerns about cost, procedures, personnel, and other issues.

Pilot studies are not simply exploratory in nature. They are designed with a clear purpose of developing some conclusions and pushing an area of research or foreshadowed problem where reformulation or the generation of other researchable questions can occur. Therefore, pilot studies should have a sound methodology before initiation.

Certain quantitative and qualitative studies have similar concerns related to sample, data collection procedures, instruments and artifacts, and data storage. The pilot study may assist in determining which type of site or sites to investigate. It answers the basic question, “Which sites are available, appropriate, and useful?” A researcher can examine and test a given type of site for inclusion in the larger study. In educational research, an example is determining which elementary school classrooms appear appropriate for examining interactive video conferencing.

The sample of interviewees or participants of interest can also be examined. The pilot study allows the researcher to practice mapping the field to locate types of individuals of interest for the later study. It helps to answer the question, “Who should I talk to or observe to get the information I desire?” In reference to the videoconference example, as the pilot study occurs, the researcher decides to interview children based on their level of engagement in the conference and employ a maximum variation sample.

Sample studies also have the added benefit of allowing the researcher to practice face-to-face interactions with the participants in order to develop interview skills and determine potential problems, such as a high mobility rate among the participants, which would decrease the possibility of a prolonged interview window. Sample studies allow the researcher to develop clear procedures for determining which individuals should be included in the study and which should not. This ability is important because the cost and time involved in determining and collecting a sample of participants can be prohibitive.

Data collection involves the actual procedures for data collection, instrument use, or equipment use, to name a few. Procedurally, a pilot study can help answer questions such as (1) how many times will interaction or contact with the participants be needed; (2) how long will these interactions take if it runs smoothly or if it does not; (3) how many interviews or observations appear to be realistic; (4) what are issues regarding ethics, anonymity, and so on of these interactions; and (5) are multiple data collectors needed and will they all need to be trained and then examined to see if they can collect the data properly? As the amount of data desired, length of the study, and type of sample increase, the need to examine and test data collection procedures becomes paramount. For example, after running a pilot study in two video conferencing classrooms and collecting fieldnotes on interactions and behaviors, the researcher realizes that a yearlong data collection model is too enormous, the data are overwhelming, and a second researcher is required.

...