Un-supervised and semi-supervised hand segmentation in egocentric images with noisy label learning

摘要

With the rapid development of wearable devices and technologies, hand segmentation remains a less explored direction in egocentric vision, which is very important for activity recognition, rehabilitation, robot self-learning, etc. To overcome the high cost of auxiliary equipment and pixel-level annotations, we present an un-supervised hand segmentation method for egocentric images. Firstly, a fully convolutional neural network (FCN) is pre-trained in source dataset containing pixel-level annotations. Then, in target dataset without labels, the network is re-trained with optimized masks, which are produced by modified local and global consistency learning (LLGC) based on pre-segmentation and superpixel features. Finally, hand segmentation is realized in an alternative way. Furthermore, to balance segmentation accuracy and the cost on labeling, we propose a new semi-supervised image segmentation framework with three subnets based on the optimized noisy masks and a small number of clean labeled data. Experimental results in two target datasets indicate that the proposed methods could achieve better performance than other methods. (C) 2019 Elsevier B.V. All rights reserved.