Comparative Analysis of Bradley-Terry and Thurstone-Mosteller Paired Comparison Models for Image Quality Assessment

摘要

In image quality assessment, preference for various image processing algorithms or treatments is often determined using paired comparisons. In this experimental design, pairs of images processed by different algorithms or "treatments" are presented to a judge. The preferred treatment is selected and a tally is kept of the number of times each treatment is preferred to another. It is possible to estimate an interval scale for treatments in a hypothetical psychological space using this method. There are two dominate paired comparison statistical models: Thurstone-Mosteller Case V (TM) (corresponding to Thurstone's Law of Comparative Judgment, Case V) and Bradley-Terry (BT). Although TM is used almost exclusively in the imaging literature, the BT formulation is more mathematically developed. Owing to its parsimony, it provides tractable maximum-likelihood estimators for scales, simultaneous confidence intervals and hypothesis tests for model fit, uniformity, and differences among populations of judges. In practice, TM and BT yield nearly identical scale estimates for complete data. In some experimental designs, many treatments are compared. Owing to the large number of possible treatment pairs, not every comparison is made, leading to an incomplete matrix of preference counts. Unlike TM, BT model applies directly to incomplete data under mild restrictions We compare and critique TM and BT models. Statistical analyses, many not available under TM, are demonstrated. An argument is made that BT offers overwhelming advantages to the imaging community and should be used instead of TM.