Is There a Role for Spin Doctors in Med Ed Research?

摘要

As an optimistic person, I understand the desire on the part of others to view the world through rose-colored glasses, see the cup as half filled, and generally look on the bright side of things. The world of medical education is replete with those who honestly view events more positively than skeptically. One might even say that optimism is an essential trait for those who toil in the medical education fields today. In the public arena, spin is associated with reframing the current story into one that is favorable to the speaker with the goal of manipulating public opinion. In fact entire careers are made through functioning as a “spin doctor.” However, medical education research suffers greatly from the influence of this spin in selling the research results in order to publish. Spin blurs the distinction between what may work and what likely will not. In this article I will define spin in education research, discuss why the practice is harmful, and propose safeguards to avoid this type of manipulation in crafting and writing education research. I can be considered an expert in this area because I violated most of these precepts early in my career. My comments pertain primarily to quantitative research where these concerns are most prevalent.;The Fishing Expedition My all-time favorite in the “spin” technique toolbox is the presentation of the sole statistically significant result in the conclusions—and title—of a research article. This is common in clinical research, even in illustrious journals, and also seen in medical education publications. Often these articles have not preselected a finite, planned number of associations for testing. Instead, the authors have looked at every possible association between known factors and the outcome of interest. The analysis does not correct for multiple associations and—voilà!—1 association is found, P??=??.04, between a medical student's score on 1 subscale out of many and the choice of pediatrics as a specialty. Only this single positive finding is presented in the title and conclusions. Although the article's limitations discuss the fact that the association may be spurious, this is buried in the Discussion section, which may not be read by all. One of my mentors, the late Dr Alvan Feinstein, termed this activity “going on a fishing expedition.” That is acceptable for pilot work, but not acceptable for a study worthy of publication. If the work has value, for example, as it reports on topics that are understudied, a finite number of associations should be planned, all results should be reported, and the study should be presented as a negative, preliminary study. I have to pause here to reflect on the term negative study. This is common parlance but not an accurate use of the word negative. These studies fail to show a difference or association that is unlikely to be due to chance (to a prespecified likelihood): The evidence supports the null hypothesis. Thus, these findings are not “negative” except in the sense that the authors may be disappointed by them. We probably need a better term.;P Values Going Solo My second favorite spin technique is the inclusion of low P values in the abstract results without the accompanying effect size, or impact. If a difference does exist between the groups studied, no matter how small, this difference can be found with a sufficiently large sample size. All other things being equal, the larger the sample size, the lower the P value. However, the P value does not reflect the size of the difference between groups. For example, in a large study of 10?000 subjects the new (and astronomically expensive) cancer drug shows a difference versus the older, generic drug with a P value?