Global motion compensation (GMC) removes the impact of camera motion andcreates a video in which the background appears static over the progression oftime. Various vision problems, such as human activity recognition, backgroundreconstruction, and multi-object tracking can benefit from GMC. Existing GMCalgorithms rely on sequentially processing consecutive frames, by estimatingthe transformation mapping the two frames, and obtaining a compositetransformation to a global motion compensated coordinate. Sequential GMCsuffers from temporal drift of frames from the accurate global coordinate, dueto either error accumulation or sporadic failures of motion estimation at a fewframes. We propose a temporally robust global motion compensation (TRGMC)algorithm which performs accurate and stable GMC, despite complicated andlong-term camera motion. TRGMC densely connects pairs of frames, by matchinglocal keypoints of each frame. A joint alignment of these frames is formulatedas a novel keypoint-based congealing problem, where the transformation of eachframe is updated iteratively, such that the spatial coordinates for the startand end points of matched keypoints are identical. Experimental resultsdemonstrate that TRGMC has superior performance in a wide range of scenarios.
展开▼
