Show Me Your Moves: Making Business as Usual the Focus of Large Customer (C/II) Self-Report Surveys

摘要

It goes without saying that separating the impacts of utility energy efficiency programs from the background adoption rates for those same technologies is a deeply challenging prospect. Given the relative absence of good quality, full-scale and longitudinal market characterization data relevant to many programs, it is still necessary to rely upon customer self-reporting techniques for at least a significant part of a net-to-gross assessment. However, there is important room for innovation to deploy self-reporting surveys that avoid many of the welldocumented pitfalls and biases of these common surveys. The authors will discuss a new technique developed in the context of a complex program serving large customers where multi-faceted and multi-party decision making is commonplace and usually recognized in the program design and program theory. The approach emphasizes and prioritizes unprompted responses from respondents where possible and puts emphasis on fully characterizing typical conditions as a point of comparison with program participation. Noteworthy steps include avoiding the use of judgment-based scalar responses and sticking to fact-based questions wherever possible, deploying an adaptive survey structure that respects respondents’ priorities and entirely skips less or irrelevant sections, tying core questions directly to the program’s logic model, and, ensuring the granularity of the scoring algorithm is appropriate to the level of confidence in the information being collected. The authors frame the new approaches within existing regulatory and professional guidance regarding best practices for self-reporting surveys.