An New Method of Audio-Visual Correlation Analysis

摘要

This paper presents a new methodology of conducting the audio-visual correlation analysis employing the gaze tracking system. Interaction between two perceptual modalities, seeing and hearing, their interaction and mutual reinforcement in a complex relationship was a subject of many research studies. Earlier stage of the carried out experiments at the Multimedia Systems Department (MSD) showed that there exists a relationship between perception of video presented in the screen and accompanying audio signals, both stereo and spatial. These results were based on subjective tests. Applying the gaze tracking system to the subjective domain may be a step towards objectivization of results obtained during such tests.In the paper, first a short review of audio-visual correlation examination methods is presented. Then, a gaze tracking system engineered at the MSD is shortly presented. The system calibration is also shown. Assumptions of the preliminary experiments are outlined, and the realization of some preliminary tests is described. Conclusions concerning the objective methodology of audio-visual correlation analysis proposed are also included.