Estimating Rater Agreement in 2 x 2 Tables: Correction for Chance and Intraclass Correlation

摘要

Many estimators of the measure of agreementbetween two dichotomous ratings of a person havebeen proposed. The results of Fleiss (1975) areextended, and it is shown that four estimators-Scott’s (1955) π coefficient, Cohen’s (1960) kˆ,Maxwell & Pilliner’s (1968) r₁₁, and Mak’s (1988)p˜-are interpretable both as chance-correctedmeasures of agreement and as intraclass correlationcoefficients for different ANOVA models. Relationshipsamong these estimators are establishedfor finite samples. Under Kraemer’s (1979) model,it is shown that these estimators are equivalent inlarge samples, and that the equations for theirlarge sample variances are equivalent. Indexterms: index of agreement, interrater reliability,intraclass correlation, kappa statistic.