Appearance based Key-Shot Selection for a Hand Held Camera

摘要

Automatic selection of key-shots is an important step for video data processing. Depending on the purpose, key-shot selection provides user feed back on recorded data, storage reduction and viewpoint selection and it can be used for panoramic image stitching and 3D-reconstruction. In particular, investigating scenes of crime or accidental investigations involves large amount of data, containing information on physical arrangement of objects, details on surface geometry and appearances. This paper proposes an efficient method for automatic selection of key-shot, providing onsite feedback on recorded segments and automatic selection of view-points for 3D-reconstruction. It uses appearance based object and scene modeling for a freely moving, hand held camera. The camera motion is determined on two levels, comparing appearances of local image regions and full 3D reconstruction. On the lower level, the 2D-warp between subsequent video frames is used to determine local change of image appearance and derive a set of motion key frames. These key-frames than are used to determine full 3D motion and to reconstruct objects. Furthermore, key-frames are used for fast indexation and detection of loop closures. Examples for automatic key-frame selection are given for an re-enacted crime scene, and compared to manual selection.