3D Scene Reconstruction from Multi-View Stereo Images Using Machine Learning

摘要

3D scene model is the basic data model in 3D GIS (Geographic Information System) which can be used for 3D geo-visualization and scene analysis. Commonly the 3D scene can be reconstructed by means of LiDAR and photogrammetry technologies, however most of the methods are time-consuming and not fully automatic. How to efficiently and automatically reconstruct the 3D scene models has become an important research issue. This paper proposes a 3D scene reconstruction method from multi-view stereo (MVS) images based on machine learning. Similar to the stereo-pair for 3D vision, the multi-view stereo mimics the human visual system (HVS) to acquire 3D information from multiple overlapping images. Because of the multiple view of an object, the problem of occlusion can be overcome. However, the complex geometric relationship between multiple view stereo images also increase the difficulty of calculation. To make the processing of 3D reconstruction more efficient and automatic, a novel method based on machine learning was introduced. Machine learning is a subset of Artificial Intelligence (AI) that provides the ability to automatically learn from data and improves from experience without too much manual intervention. Therefore, this study intends to use the advantages of machine learning to extract and train the useful features for reconstruction, improving the problems from occlusion. Based on multi-view stereo images and the machine learning model, this study aims to reconstruct the object or even the scene directly. Make the data processing operations simplified and the entire process more efficient or fully automated.