From Impressions to Data: Increasing the Objectivity of Cognitive Interviews

摘要

The use of 'think aloud' protocols is the most tangible application of cognitive psychology techniques to survey methodology - the so-called 'cognitive interview.' Think aloud methods can be used systematically but in questionnaire pretesting, the technique is usually used differently by different practitioners and relies heavily on the intuitions of the particular practitioner. This paper explores an approach to the cognitive interview that promises to increase its consistency and to provide a theoretical basis for the method. The central components of our technique are (1) a procedure for eliciting the intent of each question from question authors, and (2) a taxonomy of common respondent problems used to code think aloud protocols. By comparing the intent of the question and the content of the protocol, the researcher has relatively objective criteria for identifying where a respondent's interpretation and response behavior differ from the question author's intent, thus identifying question problems. We present a study that tests the reliability of the approach and compares it to the conventional use of the technique.