High-speed video generation with an event camera

摘要

The event camera is a kind of visual sensor that mimics aspects of the human visual system by only recording events when the light intensity on a pixel changes. This allows for an event camera to possess high temporal resolution and makes it able to capture fast motion. However, an event camera lacks information for all pixels within a scene, especially color information. In this paper, we aim to recover a typical scene in which the foreground undergoes high-speed motion which can be approximated by a planar motion, and the background is static. We demonstrate how to use the event camera to generate high-speed videos of 2D motion augmented with foreground and background images taken from a conventional camera. We match an object obtained for a static image to frames formed by the event stream, from the event camera, based on curve saliency, and we build a parametric model of affine motion to create image sequences. In this work, we are able to restore scenes of very fast motion such as falling or rotating objects and string vibration.