True communication skills assessment in interdepartmental OSCE stations: Standard setting using the MAAS-Global and EduG

摘要

? 2017 ? 2017 Background Comparing outcome of clinical skills assessment is challenging. This study proposes reliable and valid comparison of communication skills (1) assessment as practiced in Objective Structured Clinical Examinations (2). The aim of the present study is to compare CS assessment, as standardized according to the MAAS Global, between stations in a single undergraduate medical year. Methods An OSCE delivered in an Irish undergraduate curriculum was studied. We chose the MAAS-Global as an internationally recognized and validated instrument to calibrate the OSCE station items. The MAAS-Global proportion is the percentage of station checklist items that can be considered as ‘true’ CS. The reliability of the OSCE was calculated with G-Theory analysis and nested ANOVA was used to compare mean scores of all years. Results MAAS-Global scores in psychiatry stations were significantly higher than those in other disciplines (p 0.03) and above the initial pass mark of 50%. The higher students’ scores in psychiatry stations were related to higher MAAS-Global proportions when compared to the general practice stations. Conclusion Comparison of outcome measurements, using the MAAS Global as a standardization instrument, between interdisciplinary station checklists was valid and reliable. Practice implications The MAAS-Global was used as a single validated instrument and is suggested as gold standard. Background Comparing outcome of clinical skills assessment is challenging. This study proposes reliable and valid comparison of communication skills (1) assessment as practiced in Objective Structured Clinical Examinations (2). The aim of the present study is to compare CS assessment, as standardized according to the MAAS Global, between stations in a single undergraduate medical year. Methods An OSCE delivered in an Irish undergraduate curriculum was studied. We chose the MAAS-Global as an internationally recognized and validated instrument to calibrate the OSCE station items. The MAAS-Global proportion is the percentage of station checklist items that can be considered as ‘true’ CS. The reliability of the OSCE was calculated with G-Theory analysis and nested ANOVA was used to compare mean scores of all years. Results MAAS-Global scores in psychiatry stations were significantly higher than those in other disciplines (p < 0.03) and above the initial pass mark of 50%. The higher students’ scores in psychiatry stations were related to higher MAAS-Global proportions when compared to the general practice stations. Conclusion Comparison of outcome measurements, using the MAAS Global as a standardization instrument, between interdisciplinary station checklists was valid and reliable. Practice implications The MAAS-Global was used as a single validated instrument and is suggested as gold standard.