Building Robust Survey Experiments in Political Science: Or, “How I Learned to Love the Null”

Abstract

Experimental political science offers researchers a compelling way to precisely, and validly, identify causal effects. However, experiments are often plagued by concerns about their generalizability or their relevance to the “real world.” This case describes the process of designing an experiment to confirm and extend research that relied on other methodological approaches. Intended to provide confirmatory evidence of existing theory, the initial results failed to replicate previous findings in the literature. The case then describes how further tests were designed to falsify the original non-finding or null hypothesis. By the final design, the experiment featured a three-part test of U.S. citizen attitudes toward their local, state, and federal government. When no discernable differences emerged, the data spoke for themselves. Null findings, when substantiated by rigorously tested instrumentation, are findings. Even if the survey setting is limited, when situated alongside other behavioral methods, experimental research can challenge conventional theories and re-orient our attention to new paths for understanding complex social phenomenon.

locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles