Verbal Reports Are Data: A Theoretical Approach to Cognitive Interviews

摘要

The use of verbal reports to pretest questionnaires (cognitive interviews) is the most tangible outcome to date of the dialogue between cognitive psychology and survey methodology. Cognitive interviews are a standard survey pretesting tool yet they rarely exploit the theory and body of knowledge about verbal report methods. For example it is believed that if people are not aware of a thought process, they cannot verbalize it. In addition, there is evidence that verbalizing certain processes can affect the process being reported. When these are overlooked, it threatens the validity of verbal reports. We have developed a two part technique for collecting and analyzing verbal reports in cognitive interviews that takes this into account. One part concerns collecting verbal reports--the administration of cognitive interviews. The second part concerns the analysis and interpretation the verbal reports. In the data collection part, cognitive interviewers use certain generic probes when a verbalization indicates the respondent is aware, of but has not reported, useful information. In the interpretation and analysis part, coders assign segments of the verbal reports to a problem taxonomy in which a set of problem classes can be occur throughout the stages of the response process. We advocate using an approach like this to increase the validity and objectivity of cognitive interview data. Preliminary data suggests that it is promising.