Review of Research on the Ground Truth Systems for Evaluating Part Identification Systems.

摘要

In this report we discuss different types of potential ground-truth systems that could be used to test part identification systems for manufacturing assembly applications. We discuss four main ways of acquiring ground truth for evaluating part recognition/identification systems: (1) synthetic and physics based simulation, (2) manual annotation or labeling of ground-truth systems, (3) platform-based ground-truth systems, and (4) physically sensed ground-truth systems. We discuss previous efforts and discuss the different physical quantities used for ground-truth systems. In this report, we focus on static parts. The sizes of parts are in the range of 30 cm x 30 cm x 30 cm volume or smaller. The goal of parts recognition/identification is to correctly identify parts that are present in a 3D scene, usually in Red Green Blue (RGB) color and/or depth/range images. This is a challenging task especially since the scene may be cluttered, the parts in the scene may occlude each other, there could be illumination variation, and the material, reflectivity, and surface properties of parts can make it hard for perception systems to recognize or identify parts in the scene.