Evaluation of Incomplete Paired-Comparison Experiments

摘要

Incomplete paired comparison is an important technique forrncolor-imaging problems because it can avoid observers torncompare every possible pairs since the number of pairedrncomparisons for n stimuli is n(n-1)/2 which becomes prohibitivernfor large values of n. However, the experimental designer oftenrnstruggles with questions such as what is the smallest limit thernproportion of paired comparisons included that will still allowrnreliable estimations of scale values? Fortunately a Monte-Carlorncomputational simulation is carried out with a model of an idealrnobserver and the results shows that the proportion of pairedrncomparisons that is included is more critical than the number ofrnobservers who make those observations [1]. This work aims to testrnthe results from computational simulation with 25 real observersrnand 10 stimuli from the gray scale. The work suggests when eachrnobserver estimates the same proportion of paired comparisonsrnincluded the more proportion of pairs and number of observers,rnthe more accurate scale values will be produced and thernproportion of pairs is more critical than the number of observersrnwho make those observations, which quite agrees with the findingsrnfrom the computational simulation. The work also suggests whenrnthe each observer estimates a different proportion of pairedrncomparisons the more proportion of paired comparisons will notrnalways produce a more accurate scale values.