A Multi-layer Occlusion Handling Method Based on Stereo Vision and Pose Estimation for Real-time Augmented Reality

摘要

A novel multi-layer occlusion handling approach using key frames reconstruction and pose estimation was proposed for real-time augmented reality applications. Instead of calculating the stereo correspondence and dense depth map of the whole scene every frame, we combine 3D reconstruction based on stereo matching and 3D pose estimation based on feature tracking to handle multi-layer occlusion problem fastly. Information for 3D reconstruction is extracted from pairs of key frames, meanwhile a feature tracking process is running through inter frames. Models built for real objects were kept in use during the tracking process until new ones are finished. Two-threaded structure is adopted. 3D tracking for inter-frames and 3D reconstruction for key frames are running parallel in different threads. This approach needs no prior knowledge thus is fit for unknown or shape-changing surroundings and is rapid enough for real-time application. Preliminary experiments showed that the procedure time for multi-layer occlusion was shortened sharply.