Supporting interaction in augmented reality assisted industrial processes using a CNN-based semantic layer

摘要

Even though Augmented Reality (AR) is far from its maturity, we already have solutions and devices that give us an efficient technological frame in different industrial environments. Widely used mobile devices, such as tablets, or more specific ones, such as the current AR glasses available, are enough to offer solutions that improve many industrial processes; repairing, maintenance, plant control, product line reconfiguration are some examples of these tasks. Many industrial applications already utilise AR-based applications to solve those problems. In this work, we aim to go a little bit further beyond current possibilities that only focus on providing visual guidance. Our main goal is to add a semantic layer for existing AR-based applications, that visually validate worker's actions based on visual interpretation of switches, potentiometers, analog needles or buttons, among others. This semantic layer allows a new level of interaction by adding automatic interpretation of the context that affects the actions of the operator and the display of information of interest in the AR system. We propose and explain the architecture and training of the Convolutional Neural Networks (CNN) used for the semantic layer and its integration in the AR technology.