Performance Ratings: An Investigation of Reliability, Accuracy, and Relationships between Individual Differences and Rater Error.

摘要

The research reported here studied performance ratings within a person perception framework. The research program sought to: (1) discover the consistency (reliability) with which individual raters make accurate and error-free evaluations of performance effectiveness; (2) identify individual differences correlates of performance rating accuracy and of the ability to avoid halo, leniency and restriction of range rating errors; and (3) explore relationships between the number of raters providing performance evaluations of a ratee and these raters' interrater agreement and accuracy of judgment. Results of correlational analysis relating MPPB variates with accuracy and the two rating types of error showed that the individual differences variables that were measured accounted for approximately 16% of the variance in accuracy scores and about 7% in restriction-of-range scores. The largest correlates of accuracy (all positive) were verbal reasoning ability, personal adjustment, and detail orientation. A Monte Carlo study indicated that pooling ratings (across individual raters) does increase the accuracy of the ratings, though at a decelerating rate as the number of raters increases. Likewise, another Monte Carlo study indicated that to some extent high interrater reliability of a set of ratings does imply high accuracy of those ratings.