Quality Assurance Procedures in Non-Obstetric Diagnostic Ultrasound: A Study of the Reliability of Current Methods.

摘要

IntroductionudThis study aimed to address some of the issues and inconsistencies around clinical quality assurance mechanisms in (non-obstetric) diagnostic medical ultrasound. Quality assurance and resultant quality improvement in this field is sporadic with a plethora of different methodologies, techniques and quality assurance measurements tools. The evidence-base upon which programs are designed is weak with little high quality primary research in this subject area. This study aimed to clarify some of the uncertainties around clinical quality assurance mechanisms in this field of medical imaging.ududMethodologyudA website was created which allowed the retrospective review of ultrasound imaging and clinical reports to be undertaken on-line. Clinical ultrasound cases were selected which covered a wide spectrum of clinical quality and these cases were uploaded onto this site. Study participants were ultrasound practitioners invited from several professional backgrounds and levels of clinical experience who reviewed this imaging and scored the ultrasound examinations using several different quality assessment tools. ududThe on-line method of image dissemination facilitated a geographically diverse group of ultrasound practitioners to evaluate the same imaging and clinical reports using the quality assurance measurement tools provided. Outcome measurements included degree of inter-rater agreement between participants for each quality assessment tool. Systematic differences between different reviewers were also assessed.ududParticipants were given the opportunity to leave comments regarding the imaging that they had reviewed on the website if they wished. The content and tone of these comments was also analysed.ududResultsudThe inter-rater agreement was classed as ‘fair’ for all the quality assurance tools under investigation. There was no significant difference between any of the quality measurement tools in terms of inter-rater agreement. Correlation between tools was good. ududThere were weak systematic differences found between reviewers. Practitioners of more clinical experience rated image quality more highly than those of lesser clinical experience. Practitioners of lower clinical grade tended to rate the quality of clinical report more highly than those of a higher clinical grade.ududParticipant comments were evenly divided between comments on clinical technique and comments on the quality of the written report. An ultrasound specialist judged that ‘expert-group’ participants were more likely to give constructive comments than a ‘peer-group’ of reviewers, but this finding was not confirmed when the comments were analysed by a non-specialist in clinical ultrasound. Overall, there were slightly more constructive comments than non-constructive, but a large proportion of the comments were judged to be non-constructive in nature.udConclusionsudThe study demonstrated significant inter-rater variation in quality assessment of diagnostic ultrasound which is probably inherent within the imaging modality itself. Efforts should be directed to managing this variation rather than attempting to eradicate it.udThere are some systematic differences between study participants but there was insufficient data to accurately model the precise systematic effects of different participant characteristics and this requires further research with a larger cohort of study participants.udThere is scope to improve the quality of feedback to ultrasound practitioners, particularly when this is subjective in nature to maximise the probability of this resulting in positive subsequent change. Formal tuition in the theory and practice of giving feedback should be available to all staff undertaking quality assurance work, irrespective of their degree of expertise, clinical grade or clinical experience.ududRecommendations for Clinical Practice.udThe following clinical recommendations have been made, based on the evidence gained from this study;udud•Quality assurance of non-obstetric ultrasound examinations should ideally be undertaken by those of a higher clinical grade than those undertaking the work being appraised. Peer audit may be acceptable providing there is effective oversight by a senior clinical practitioner.ududud•The use of a single, expert reviewer in diagnostic ultrasound does not provide adequate assurance in terms of inter-rater reliability and therefore should not be used. A quality assurance program based on retrospective assessment of ultrasound imaging and clinical reporting should be undertaken by several reviewers to buffer against the effects of inter-reviewer variation.udud•Use of the internet provides great advantages in terms of overcoming logistical difficulties in undertaking quality assurance in ultrasound, particularly when undertaken by an external reviewer. However, individual feedback should be given to ultrasound practitioners face-to-face by the senior practitioner responsible for the quality assurance program.udud•Those undertaking quality assurance work, regardless of clinical grade and expertise should receive formal training in giving feedback in a constructive fashion. The purpose is to maximise the potential for this feedback to lead to improved clinical standards and outcomes for patients.udud•There is currently no primary research evidence to favour one quality assurance tool over another. Quality assurance tool selection may therefore be done at an individual ultrasound unit level. In selection of a quality assurance tool, ultrasound units should consider which tool best reflects the individual requirements and workload of that unit.