Evaluating Econometric Evaluations of Post-Secondary Aid

摘要

The question of whether and how financial aid affects college enrollment remains central in discussions of higher education policy. Most econometric investigations of this question identify causal effects using non-experimental strategies such as covariate conditioning, differences-in-differences panel methods, and regression discontinuity (RD) designs. The resulting empirical analyses have produced a wide range of estimates, perhaps reflecting the diversity of the models and assumptions used in this work (see research surveyed in Deming and Dynarski 2009). In an effort to produce credible and robust estimates of the causal effects of aid on post-secondary outcomes, we've worked with the Susan Thompson Buffett Foundation (STBF) to conduct a randomized evaluation of STBF's longstanding scholarship program for Nebraska high school seniors. Findings to date are detailed in our working paper Angrist et al. (2014).Our focus here is methodological: in the spirit of LaLonde's (1986) pioneering comparison of job training effects from randomized and non-experimental analyses, and the recent Wing and Cook (2013) within-study evaluation of a RD design, we compare our experimental results with covariate-controlled estimates from a pre-experimental cohort and with RD estimates from the experimental sample. The results show covariates do little to mitigate selection bias, but RD estimates that exploit institutional idiosyncrasies in the award process come close to an appropriately-defined experimental benchmark. On the other hand, the RD estimates are sensitive to controls for the running variable.