Image-based state tracking in Augmented Reality supported assembly operations

摘要

Visual tracking and holographic information representation techniques have become robust enough to support operators in complex tasks on the shop floor. This paper presents an approach for coupling AR-supported assembly task instructions with image-based state tracking, so as to assist the operators in product assembly operations. The developed system consists of a visualization platform for AR-supported assembly instructions, a state tracker that includes object recognition, localization and hand tracking, using deep neural networks, and a server that handles the data exchange between the two. The developed framework is applied and validated in an industrial use case.