To Adjust or Not to Adjust Alpha in Multiple Testing: That is the Question. Guidelines for Alpha Adjustment as Response to O'Keefe's and Matsunaga's Critiques

摘要

The pro and con arguments for alpha adjustment in multiple testing have been sufficiently reviewed by Matsunaga (this issue), O'Keefe (2003a, 2003b), Hewes (2003), and Tutzauer (2003). Based on these arguments, O'Keefe concludes that alpha adjustment should be generally rejected as long as there is no "principled basis" for it. Matsunaga, however, argues for a better definition of what constitutes a family of null hypotheses and promotes the use of false discovery rate control procedures as an alternative. Since both authors reason differently and some of their conclusions are based on incorrect or incomplete premises, clarification is necessary. This paper tries to resolve this ambiguity by providing guidelines for alpha adjustment and uses these guide-lines for a review of Matsunaga's and O'Keefe's notion of the alpha adjustment problem.