Assessing the Usefulness of a New Measure of Interviewer Performance in Telephone Surveys

摘要

This research note compares the properties of a new measure of interviewer performance—the Net Contribution to Performance Index (NCPI) (Durand, 2005a)—with the most commonly used measure for telephone surveys, cooperation rate at first contact (Cooprt1), to assess their reliability, validity, and usefulness in various contexts. The two measures are compared in terms of their relationship with cooperation and response rates at the survey level, their relationship with mean interview length, their relationship with one another in different survey contexts and their ability to reliably trace variations in interviewer performance over time. The data come from 14 RDD polls conducted by three different pollsters and a governmental statistical institute on various topics with response rates (AAPOR-RR1) ranging from 21.3 to 52.7 percent. The NCPI has some advantages over Cooprt1, but more so for polls in which obtaining cooperation is difficult and high response rates are sought. Compared to Cooprt1, NCPI allows for the measurement of performance whatever the task performed; it has no missing values. It is not related to interview length. It is related to survey response rates and cooperation rates. It traces the daily evolution of each interviewer's performance where Cooprt1 seems to vary at random or be too stable. It appears to be a better and more useful measure of interviewer performance than Cooprt1. It seems important to continue research to assess whether using NCPI could lead to different conclusions when experiments aimed at improving interviewer performance are conducted.