Commentary: For alt.chi paper The Continued Prevalence of Dichotomous Inferences at CHI

摘要

The following is my unaltered review for the original submission. Great work. One of the things that struck me while reading the paper is the issue who/what contributes to the continuance of reporting dichotomous outcomes? Often, but not always, the choice to report dichotomous outcomes reflects the authors' true desire. Sometimes, however, the author would prefer to NOT report dichotomous outcomes (for good reasons), but is compelled to do so by their fear/knowledge that if not included, reviewers will expect it and criticise its absence (I've certainly succumbed to this in past papers). Other times, the authors stick by their convictions and choose not to brand outcomes as "sig./not sig.", but get beaten up by reviewers for following through with their choice... then, in rebuttal authors can stick with their convictions to not report (which is likely acceptance suicide) or bow to the reviewers' "wisdom" and include it (elevating acceptance probability) - I've fallen victim to this on both sides.