Reconsidering Yarbus: Pattern classification cannot predict observer's task from scan paths

摘要

A very familiar illustration by Yarbus shows dramatic differences in eye movement patterns when a viewer performs different tasks while viewing the same image. The scan paths seem to be windows into the observer's mind but can the intentions of the viewer really be read from the pattern of eye movement. Yarbus' data are qualitative, drawn from only one observer examining one image. We showed 64 photographs to 16 observers for 10 s each while eye movements were recorded and observers performed one of four tasks: memorize the picture, determine the decade in which the image was taken, determine how well people in the picture knew each other, or determine the wealth of the people. Eye movement data were fed into a pattern classifier to predict task, using leave-one-out training and testing. Although the classifier could identify the image at above chance levels (23% correct, chance = 1.6%) as well as the observer (31% correct, chance = 6.3%), it was at chance identifying the task (28% correct, chance = 25% p = .49). Perhaps the earliest eye movements held the predictive information? Examining the first 2 and 5 seconds also yielded chance classification performance (27.4% and 27.7% correct). Perhaps more viewing would be more predictive? 16 additional participants viewed the images for 60 seconds each. Classifier performance remained at chance (28.1%). So, perhaps we built a bad classifier. Surely human observers can use observers' patterns of eye movements to predict task? 20 observers viewed another observer's eye movements, plotted over the image, and tried to predict which task was being done. Participants were at chance with either 10s (27.4%) or 60s (27.5%) scan paths. The famous Yarbus figure may be compelling but, sadly, its message appears to be misleading. Neither humans nor machines can use scan paths to identify the task of the viewer.