Rigor

Rigor in Different Research Stages

Investigators need to make various methodical commitments through the whole research process, from methodology selections to data collections, data analysis, and data interpretations and presentations. At each research stage, a structured and controlled manner of planning, conducting, and analyzing the study has to be followed to make sure that the study is rigorous.

First, investigators need to decide research questions or hypotheses. After the research questions or hypotheses have been decided, investigators start to consider the following questions: what is the appropriate methodological tool for the research questions/hypotheses: a qualitative approach or a quantitative one? Which specific method to use: experiments, surveys, interviews, field studies, case studies, content analysis, or a combination of different methods? Can this methodological tool meet the project’s objectives?

An appropriate methodological tool is the first step to attain rigor in a study. In the next step, the rigor will be reflected in how attentive the investigators are to selecting a sample. The investigator needs to consider the advantages and limitations of various sampling strategies and choose a strategy and a sample size that meet research goals. If the research is a quantitative study, investigators would want to select a sample that can maximize the generalizability of the study. If the research is qualitative, investigators would try to identify the most interesting cases and maximize the chance of identifying the full range of phenomenon of interest. Besides finding representative respondents, the investigators would need to make other sampling decisions as well, including the following: When to conduct the study? Which season or which time of the day? At what locations? For how long? For example, if the research question is how many coffees do college students drink per week, then the answers to the question will vary depending on when in the semester the researcher conducts the study. The researcher can anticipate different answers at the beginning of the semester than immediately before final examinations.

[Page 1512]After the sampling strategies have been set, investigators will start to collect data. During this stage, investigators would need to meticulously carry out the research to make sure that the data are detailed, abundant, and unbiased. The collected data shall have enough details for answering the research questions or hypotheses. Researcher biases and other subjective influences should also be excluded.

When a data analysis stage starts, investigators will decide on analytic techniques. For a quantitative study, investigators need to ask themselves the following questions: How general or specific do we need to analyze the data? How statistically significant? How will the analytic techniques help to interpret the results? For a qualitative research, investigators need to consider how to ensure the discovery of all salient themes and topics and maximize the potential for finding relationships among themes and topics. During the final stage of writing and presenting the results, rigor requires investigators to strictly adhere to the data. The results and interpretations are expected to be comprehensive and unbiased.

...