Associative Segmentation for Instances and Semantics by Perceiving Neighborhood in Point Clouds

摘要

In the process of human visual perception, when facing complex scenes, people often rely on the neighborhood information of an object to aid in understanding scene, which is applied to robot visual perception as well. In this paper, we propose a neighborhood-aware module (NAM) which captures rich instance-level contextual dependencies to reduce the search space of possible categories for segmentation tasks. To address the instance and semantic segmentation associatively, we further design a Associative segmentation module (ASM) to make two tasks promote each other and get a win-win solution. Experimental results on two well-known dataset (S3DIS and ShapeNet) show that our NAM is capable of caturing contextual dependency and ASM boosts the performance by enabling semantic segmentation and instance segmentation to take advantage of each other. Our method largely outperforms the state-of-the-art methods in 3D instance segmentation, as well as achieving a significant improvement in 3D semantic segmentation.