Kappa Coefficient of Agreement

Kappa, one of several coefficients used to estimate inter-rater and similar types of reliability, was developed in 1960 by Jacob Cohen. In its original conception, kappa, denoted κ, was an index used to measure the level of consistency between two raters who use rubrics or other instruments to place subjects (i.e., people) into one of κ nominal categories. For example, two evaluators might use a rubric to classify the instructional strategies used in a classroom as one of two nominal categories, such as effective or ineffective, or two psychologists might use an instrument to identify a person’s depression as major depression, bipolar disorder, persistent depression, or psychotic depression. In both of these cases, subjectivity might lead to disagreements about the category assigned and raise concerns ...