For Trial Programs, It’s Not All about Energy Savings: A Methodology for Evaluating a Retail Plug Load Program Trial

摘要

California utilities have aggressive goals for achieving energy savings from consumer electronics and home appliances. While declines in per-unit energy consumption are evident with many product categories, the number of these “plug load” products in residences is growing, threatening the cost effectiveness of typical rebate-based programs. In response, a new generation of midstream program designs provides an opportunity to move beyond traditional “widget-based” rebate approaches by aggregating small per-unit savings across a large number of units, but these program designs are challenging to evaluate. This paper presents the evaluation methods being used for the new Retail Plug Load Portfolio Program Trial being conducted by Pacific Gas and Electric Company and Sacramento Municipal Utilities District. The trial provides per-unit incentives to a retailer for sales of qualified products. Energy savings for the program will be measured by changes in the primary metric—sales-weighted unit energy consumption within product categories. However, given the trial’s small scale and short one-year duration, detecting energy savings may be challenging. But changes in the primary metric are only one indicator of program success. To assess the range of potential outcomes and impacts of the program, we leverage multiple methodologies to: (1) validate program theory, (2) measure quantitative and qualitative primary and proximate metrics, and (3) inform future evaluation methods for a full-scale program rollout. This paper discusses these multiple methods in detail and provides insights into how the effectiveness of future program designs can be tested using a “preponderance of evidence” approach.