Selection of Reliable and Valid Surgeon Performance Measures

摘要

Objective- To identify measures of surgeon performance that are valid, reliable, and capable of classifying the risk of surgeon performance. Data Sources- A surgical quality improvement program, dataset unique to selected hospitals and surgeons containing abstracted surgical case records. Study Design- Six criteria were employed to assess the validity of 24 candidate measures of surgeon performance: 1) the presence of a surgeon random intercept; 2) a surgeon signal that is greater than zero; 3) surgeon majority control; 4) reliability of the surgeon random intercept of at least 0.7; 5) the capacity to identify both low- and high-risk surgeons and 6) the presence of a learning/improvement effect. Data collection/Extraction methods- Surgical case review nurses abstracted cases for each surgeon using a structured sampling and abstraction methodology. Principal findings- Comparing outcomes requires risk adjustment and the use of the "true score" approach but is limited by case volume constraints and a confounding factor, i.e., the hospital, if used to judge surgeons' performance. Assessing surgeon performance requires a measure of the surgeon's effects on the consequences (postoperative occurrences) of surgical procedures, i.e., the surgeon-specific random intercept, which is a product of a multilevel risk adjustment model. Conclusion- Morbidities and mortality lack the characteristics necessary to be used as measures of surgeon performance. However, the process (task-time) measures LOS and OT both have high event rates, high reliability, and are capable of classifying surgeon risk.