Validity Coefficients

Interpreting Validity Coefficients

Validity coefficients can be described with respect to direction and magnitude and can vary between −1 and +1. A validity coefficient of 0 indicates that there is no linear relationship between scores on the measure and scores on the criterion variable. Such a correlation has no magnitude and no [Page 1781]direction. A coefficient of .90 has a large magnitude (a value of 1 would be largest possible magnitude) and has a positive direction such that as one variable increases (e.g., a reading readiness test score), the other variable increases (e.g., reading performance in the classroom). A coefficient of −.15 has a small magnitude and a negative direction such that as one variable increases (e.g., school days absent), the other variable decreases (e.g., family socioeconomic status). A validity coefficient describes a relationship, but it does not necessarily imply causation.

The closer the validity coefficient is to 1, the more confident one can be in the accuracy of the inferences drawn from scores on the measure. For example, if a test designed to predict success in college has a validity coefficient of 1, it means that the test has a perfect linear relationship with the criterion of interest (e.g., freshman year grade point average). In other words, scores on the test perfectly predict freshman grade point average, thus the higher an individual’s score on the test, the higher the individual’s future grade point average. In this situation, test users (i.e., college admissions staff) can feel very confident that they can make accurate decisions based on test performance. If the validity coefficient of this test was 0, it would indicate that test performance does not accurately predict college success and should, therefore, not be used.

Much of social science research relies on Jacob Cohen’s guidelines for interpreting the magnitude of a correlation. He suggested that correlations of .1 represent a small effect, correlations of .3 represent a moderate effect, and correlations of .5 or greater represent a large effect. Using these guidelines, a validity coefficient of .35, though far from a perfect correlation of 1, may be considered useful. In fact, any nonzero correlation provides some predictive value. Although these guidelines can be useful, they do not provide insight into the practical impact of using the measure. Therefore, other methods, such as Taylor–Russell tables, utility analyses, and tests of sensitivity and specificity, are also commonly used.

In the field of personnel selection, Taylor–Russell tables use the validity coefficient, selection ratio (number selected or hired), and the proportion of candidates that would perform well on the criterion variable to determine the probability that a candidate who performs well on a selection test will perform well on the job. The results of the analysis provide test users with an estimation of how much using the test improves their hiring decisions as compared to not using the test. Utility analysis builds on the Taylor–Russell tables by considering the monetary impact of using the test to make hiring decisions. Utility analyses can be helpful for organizations deciding whether or not to use a test for selection purposes because they provide the organization with a cost versus benefit estimation.

...