Combined User Physical, Physiological and Subjective Measures for Assessing User Cost

摘要

New technologies are making it possible to provide an enriched view of interaction for researchers using multimodal information. This preliminary study explores the use of multimodal information streams in evaluating user cost. In the study, easy, medium and difficult versions of a game task were used to vary the levels of the cost to user. Multimodal data streams during the three versions were analyzed, including eye tracking, pupil size, hand movement, heart rate variability (HRV) and subjectively reported data. Three findings indicate the potential value of multimodal information in evaluating usability: First, subjective and physiological measures showed significant sensitivity to task difficulty. Second, different user cost levels appeared to correlate with eye movement patterns, especially with a combined eye-hand measure. Third, HRV showed correlations with saccade speed. These results warrant further investigations and take an initial step toward establishing usability evaluation methods based on multimodal information.