Designing a Quasi-Experimental Study to Test the Community College Penalty Using Propensity Score Matching Methods

Abstract

We present a case study of the process through which a methodology was developed and applied to a quasi-experimental research study that employed propensity score matching. Methodological decisions are discussed and summarized, including an explanation of the approaches selected for each step in the study as well as rationales for these selections. Examples include identification and creation of ‘treatment’ and ‘control’ groups, application of relational database software and methods, calculation of propensity scores, accounting for multilevel effects, post-treatment changes and identification of post-treatment adjustment, and selection of a propensity matching algorithm. We demonstrate that much of the propensity score matching process focuses on creating a valid counterfactual or control group. Thus, propensity score matching allows researchers to focus on creating conditions that help show the impact of the treatment, rather than on other factors that may be related to the outcome of interest. Additional items discussed include decisions about missing data, use of balancing diagnostics, determination of the effect of the treatment on the outcome of interest, and sensitivity analysis. The authors propose that an appropriate methodology for such a study is best arrived at through an iterative, experimental process.

locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles