Why Health Care Process Performance Measures Can Have Different Relationships to Outcomes for Patients and Hospitals: Understanding the Ecological Fallacy

摘要

Relationships between health care process performance measures (PPMs) and outcomes can differ in magnitude and even direction for patients versus higher level units (e.g., health care facilities). Such discrepancies can arise because facility-level relationships ignore PPM–outcome relationships for patients within facilities, may have different confounders than patient-level PPM–outcome relationships, and may reflect facility effect modification of patient PPM–outcome relationships. If a patient-level PPM is related to better patient outcomes, that care process should be encouraged. However, the finding in a multilevel analysis that the proportion of patients receiving PPM care across facilities nevertheless is linked to poor hospital outcomes would suggest that interventions targeting the health care facility also are needed. Many health care performance measures (PMs) quantify the extent to which a process of care that has been shown to cause or predict positive outcomes among participants in empirical studies is applied to patients in health care facilities (e.g., hospitals). 1 , 2 Such process performance measures (PPMs) are implemented on the assumption that processes of care linked to positive patient outcomes in clinical trials and other research will be associated with positive facility-level outcomes when the PPM is aggregated to the facility level as the proportion of patients receiving the PPM care. For example, it might be assumed that if coordinated care for a particular medical condition is linked to better outcomes for patients in randomized controlled trials, then health care facilities with higher levels of coordinated care for targeted patients should have higher proportions of patients with good outcomes. However, researchers who have investigated hospital- or facility-level PPM–outcome relationships sometimes have found that facility rates of PM-specified care are unrelated or only weakly related to facility-level outcomes. One example was reported by Bradley et al., 3 who examined National Quality Forum PPMs for treating patients with acute myocardial infarction. They found that higher rates of provision of the practices recommended by the National Quality Forum were at best only modestly related to lower hospital-level, risk-adjusted 30-day mortality rates among acute myocardial infarction patients from more than 900 hospitals. Werner and Bradlow 4 conducted a more comprehensive analysis of data from approximately 3600 acute care hospitals. Their findings showed that facilities in the top and bottom quartiles in terms of proportion of patients receiving processes of care recommended (by the Centers for Medicare & Medicaid Services and the Joint Commission on Accreditation of Healthcare Organizations) for acute myocardial infarction, heart failure, and pneumonia differed only slightly in risk-adjusted 30-day and 1-year mortality rates. (At the facility and practice level, Lehrman et al. 5 and Sequist et al.6 also found weak relationships between clinical care quality and patient satisfaction.) Bradley et al.3 and Werner and Bradlow4 pointed to a variety of factors that may have accounted for the weak facility-level associations, including potential facility-level confounding factors (e.g., patient safety processes) and restricted variation across hospitals in the provision of certain practices (e.g., providing aspirin at admission to patients with acute myocardial infarction symptoms). Although Werner and Bradlow called for PPMs that are more strongly related to patient outcomes, neither they nor Bradley et al. considered that, even though facility-level performance on these care processes was only weakly associated with aggregated facility outcomes, patients who received this type of care may have had significantly better outcomes than patients who did not. We considered this apparent paradox in the context of the methodological literature on the ecological fallacy and cross-level bias that have been the focus of considerable work in such fields as epidemiology and sociology, 7 – 15 as well as the highly relevant statistical literature on multilevel analysis 16 – 18 that has been applied to address other issues in health care research, 19 – 21 including linking patient care processes to outcomes. 22 However, these issues have received scant attention in the quality literature on PPM–outcome relationships. 3 – 6 We used data on a PM for treatment retention among patients with substance use disorders to examine differences in patient- and facility-level PPM–outcome relationships. 23 We then examined how the literature on cross-level bias and multilevel analysis can explain otherwise puzzling differences in PPM–outcome relationships at different levels of analysis. Our goal was to explain these issues in a nontechnical way that is access