A Validation Study of Self-Reported Behavior: Can College Student Self-Reports of Behavior Be Accepted as Being Self-Evident?

摘要

The dearth of institution-reported data describing college student behaviors forces higher education researchers to rely upon self-reported behaviors collected via surveys and summarized by the survey statistic. If bias is present and substantive, the invalid, self-reported input data threatens the validity of research conclusions that guide co-curricular and academic policies and best practices at institutions of higher education. This study evaluates the premise of self-reported data validity upon which survey research is based. Potential sources of bias include (1) response bias from inaccurate self-reporting and (2) nonresponse bias where characteristics of respondents differ from those of nonrespondents. The assumption of survey statistic validity depends upon it being an unbiased estimate of behavior in a target population that relies on survey responses being free from systematic biases, an assumption that has not been adequately tested in the literature. This validation study of self-reported behaviors compares institution-reported, transactional data to corresponding self-reported academic performance, class attendance, and co-curricular participation from a sample of 6,000 students, using the Model of the Response Process by Tourangeau (1984, 1987). Response bias, observed as measurement error, is significant in 11 of the 13 questions asked and evaluated in this study. Socially desirable behaviors include campus recreation facility (CRF) use and academic success being overstated as much as three times. Nonresponse bias, observed as nonresponse error, is also significant in 11 of the same 13 questions asked and evaluated with high GPA and participatory students over represented in the survey statistic. For most of the questions, measurement error and nonresponse error combine to misstate behavior by at least 20%. The behaviors most affected are CRF use, which is overstated by 112% to 248%; semester GPA self-reports of 3.36 versus an actual value of 3.04; and co-curricular participation that misstated by between -21% to +46%. This validation study sufficiently demonstrates that measurement error and nonresponse error are present in the self-reported data collected for the commonly studied topics in higher education that were represented by the 13 questions. Researchers using self-reported data cannot presume the survey statistic to be an unbiased estimate of actual behavior that it is generalizable to larger populations.