Detecting topical events in digital video

摘要

The detection of events is essential to high-level semantic querying of video databases. It is also a very challenging problem requiring the detection and integration of evidence for an event available in multiple information modalities, such as audio, video and language. This paper focuses on the detection of specific types of events, namely, topic of discussion events that occur in classroom/lecture environ-ments. Specifically, we present a query-driven approach to the detection of topic of discussion events with foils used in a lecture as a way to convery a topic. In particular, we use the image content of foils to detect visual events in which the foil is displayed and captiured in the video stream. The recognition of a foil in video frames exploits the color and spatial layout of regions on folis using a technique called region hasing. Next, we use the textual phrases listed on a foil as an indication of a ctopic, and detect topical audio events as places in the audio track where the best evidence for th topical phrases was heard. Finally, we use a probabilistic model of event likelihood to compine the results of visual and audio event detection that exploits their time cooccurence. The resulting identification of topical events is evaluated i nthe domain of classroom lectrues and talks.