Determining Computable Scenes in Films and their Structures using Audio-Visual Memory Models

摘要

In this paper we present novel algorithms for computing scenes and withinscene structures in films. We begin by mapping insights from film-making rules and experimental results form the psychology of audition into a computational scene model. We define a computable secne to be a chunk of audio-visual data that exhibits long-term consistency wiht regard to three propertis: (a) chromaticity (b) lighting (c) ambient sound. Central to the computational model is the notion of a causal, finite-memory viewer model. We segment the audio and video data separatelyh. In each case we determine the degree of correlation of the most recent data in the memory with the past. The respective scene boundaries are ddetermined using local minima and aligned using a nearest neighbor algorithm. We introduce a periodic analysis transform to automatically determine the structure within a scene.