Video Scene Retrieval with Symbol Sequence Based on Integrated Audio and Visual Features

摘要

In this paper, we propose a method to retrieve semantically similar scenes to a query video from large scale video databases at high speed. Our method uses the audio features and the color histogram as the visual feature because the audio signal is closely related with the semantic content of videos and the color is an extensively used feature for content-based image retrieval systems. The feature vectors are extracted from video segments called packets and clustered in the feature vector space and transformed into symbols that represent the cluster IDs. Consequently, a video is expressed as a symbol sequence based on audio and visual features. Quick retrieval of similar scenes can be realized by symbol sequence matching. We conduct some experiments using audio, visual, and both features, and examine the effect of each feature on videos of various genres.