Testing the consistency (or lack thereof) between choices in best-worst surveys

摘要

Best-Worst (BW) stated choice surveys (SC) have rapidly increased in popularity in recent years, in fields as diverse as transport, marketing and health research. They provide more information per choice task, and are considered superior to standard ranking due to a perceived propensity by respondents to identify and respond more consistently to extreme options. Despite numerous applications of different variants of BW SC, the stability of preferences across stages has not been tested in any rigorous manner to date, with analysts relying on an assumption that only differences in scale exist between stages. This paper fills the gap by investigating the stability of response patterns across stages for sequential and simultaneous variants of BW SC. Using datasets from three different studies, we show that, regardless of the dataset used, the obtained estimates are not stable across stages, going beyond the differences in scale commonly taken into account. In all our datasets, we found significant and substantial differences between marginal utility estimates and implied monetary valuations across stages.