Observer Reliability

Observer reliability is an important consideration for researchers interested in content analysis, coding open-ended survey responses, or coding spoken words observed between two or more communicators. Defined, observer reliability is the degree to which a researcher’s data represents communicative phenomena of interest, or whether it is a false representation. In other words, observer reliability is a defense against observations that are superfluous. A procedure is reliable to the extent that it yields identical results regardless of the context or external circumstances at the time of data collection and analysis. In many ways, observer reliability is analogous to how well a group of individuals are able to consistently hit the bull’s eye on a dart board. A researcher can confidently claim his or her ...