Reliability

Definitions of Reliability

Reliability is a concept derived from quantitative research. There, a reliable measure is one that yields consistent results. It may be described as consistent or repeatable. Reliability is often contrasted with validity. A valid measure is one that measures what it claims to. An example often given is that a reliable clock, in the technical sense of the term, is one that is consistent, even if always fast or slow by a fixed amount. On the other hand, a clock is akin to an adequately valid measure when it always very nearly indicates the correct time, even if sometimes a little fast or a little slow. Together, reliability and validity provide accuracy. A measure that is very unreliable cannot be valid. Reliability without validity is not useful.

Yvonna Lincoln has proposed the use of the terms dependability and consistency in qualitative research as replacements for reliability. In some literature, including some action research literature, this has become common usage. Lincoln has since advanced stability as a more appropriate term, less indebted to positivist notions of good research. In consequence, there is no commonly agreed-on terminology for the concept in qualitative research.

Reliability can also be applied to procedures or processes, where it is the characteristic of being repeatable. A reliable research procedure delivers the same result on the same sample at different times or on different but equivalent samples. In their much cited monograph, Jerome Kirk and Marc Miller call these ‘diachronic reliability’ and ‘synchronic reliability’, respectively. For procedures or measures, reliability is often reported as a correlation coefficient or sometimes as the percentage of agreement between the results being compared.

Quantitative Measures in Action Research

Although most action research is qualitative, quantitative measures may be incorporated as part of an action research study. This is not unusual, for example, in some community-based participatory research. A team consisting of researchers and community members may [Page 684]administer quantitative measures perhaps to determine what the focus of the action research is to be or for diagnostic purposes. The administration of the measure and the collection and scoring of data follow quantitative research protocols. Researchers and research participants then determine collaboratively the local relevance of the scores and their practical implications. For such quantitative components, traditional definitions of reliability apply. High reliability is achieved by adopting the practices of good quantitative research. This is likely to include strategies such as standardized administration, judicious random sampling and careful recording of results.

...