Semantic segmentation method of 3D reconstructed model using incremental fusion of 2D semantic predictions

摘要

The present invention is a three-dimensional restoration model through gradual mixing of two-dimensional semantic segmentation information that performs three-dimensional restoration from a continuous color and depth image stream from a diffusion depth image camera and gradual semantic segmentation of the reconstructed model. In the semantic segmentation method of the three-dimensional reconstruction model according to the present invention, the semantic segmentation method of (a) a depth image corresponding to the color image (RGB) of each input image (Depth ) To perform deep learning-based semantic segmentation by pixel to obtain probability information according to an object class for each pixel; (b) updating the probability information obtained for each pixel in the voxel grid by raycasting; (c) extracting a mesh model from a voxel grid by a marching cube algorithm; And (d) performing semantic division of the 3D reconstructed model by selecting a class having the highest probability for each vertex in the mesh model.