On the Repeatability of Paired Comparison Based Scaling Methods

摘要

Paired comparison based scaling method such as the method of paired comparison and the method of ranking, etc, are often used for subjective image quality measurement. The scaled values obtained are often used to predict customer preferences and are used for modeling purposes. Knowing the repeatability of these measurements is therefore important. This paper shows that the theoretical repeatability of paired comparison based subjective measurement can be simulated numerically. Also, in this paper, paired comparison based image quality measurement or scaling data are used to examine the potential repeatability of these measurements by sub-sampling the measured observer pool. In practice, it is strongly desirable to have as small a number of observers as possible. This type of analysis can help to reveal the potential risk when a small number of observers are used.