Comparing Resident Self-Report to Chart Audits for Quality Improvement Projects: Accurate Reflection or Cherry-Picking?

摘要

What was known Residents' adherence to prevention and screening guidelines for ambulatory care is an important area for ambulatory quality improvement (QI).;What is new Resident self-review and reporting overstated adherence to guidelines, compared to external chart review.;Limitations Single institution, single specialty, and potential for sampling bias limit ability to generalize from the findings.;Bottom line Ongoing external review and improving resident education about chart review could enhance the reliability and utility of chart self-audits as a QI tool.;Editor's Note: The online version of this article contains preventive care guidelines used for the study.;Introduction Adherence to national health care guidelines may be as low as 50%,1 and initiatives to increase compliance could save $150 billion in health care spending annually.2 The Institute of Medicine recommends that quality improvement (QI) instruction begin in residency and continue throughout practice,3 and the Accreditation Council for Graduate Medical Education identified a QI curriculum as a residency program requirement across all specialties.4 How to best incorporate QI remains undetermined. Chart audits have proven effective at improving resident adherence to primary care guidelines5,6; however, recruiting an appropriate auditor can be difficult and resource intensive.7 Previous investigators have advocated residents as the auditors of their own charts to increase investment in quality patient care, but the impact on audit validity is undefined.8 Self-reported questionnaires and face-to-face interviews have demonstrated that both residents and attending physicians overstate clinical performance, compared to independent chart review.9–11 Although our program hoped to minimize bias by anchoring reported performance to specific patients, the validity of these self-audits remains unknown. Residents could bias their results by preferentially selecting (either consciously or unconsciously) patients who received guideline-adherent care. The purpose of our study was to determine if resident-selected chart audits were a reliable means of assessing patient care performance. We compared adherence to primary care guidelines from chart reviews selected by residents on their own patients (self-audits) to those audits of charts randomly selected by an external reviewer (external audits).;Methods Study Design The Penn State Hershey Medical Center Internal Medicine Residency Training Program recruits 20 categorical residents per year, with primary care clinic 1 half-day per week for each resident. During the 2010–2011 academic year, clinic records consisted of electronic physician notes, as well as a paper record documenting patient problems, medications, and vaccinations. Our curriculum includes periodic audits performed by resident physicians of their primary care patient charts. For 6?months, residents were instructed to randomly select 10 patient charts to review and report guideline adherence based on a subset of 16 US Preventive Services Task Force (USPSTF) guidelines (provided as online supplemental material).12 There were no specific instructions on chart selection. Residents' chart audit results were submitted to attending preceptors, and residents were charged with creating a QI plan to improve their performance on subsequent reviews. In presenting this assignment, emphasis was placed on the creation of a QI plan over strict adherence to guidelines. Although residents received instruction in the USPSTF guidelines, there was no specific training in their application to this initiative. After receiving approval from the Institutional Review Board, we randomly selected independent audits for second- and third-year residents during the 2010–2011 academic year. We excluded first-year residents, who had not established care with sufficient patients to assess adherence. The external auditor (K.T.) was a medical student instructed in USPSTF guidelines and usage of th