Robust Video and Audio Based Synchronization of Multimedia Files

摘要

This paper addresses the problem of robust and automated synchronization of multiple audio and video signals. The input signals are from a set of independent multimedia recordings coming from several camcorders and microphones. While the camcorders are static, the microphones are mobile as they are attached to people. The motivation for synchronization of all signals is to support studies and understanding of human interaction in a decision support environment that have been limited so far due to the difficulties in automated processing of any observations during the decision making sessions. The application of our work is to environments supporting decisions. The data sets for this work have been acquired during training exercises of response teams, rescue workers, and fire fighters at multiple locations.rnThe developed synchronization methodology for a set of independent multimedia recordings is based on introducing aural and visual landmarks with a bell and room light switches. Our approach to synchronization is based on detecting the landmarks in audio and video signals per camcorder and per microphone, and then fusing the results to increase robustness and accuracy of the synchronization. We report synchronization results that demonstrate accuracy of synchronization based on video and audio.