Computer Vision Systems and Methods for Unsupervised Representation Learning by Sorting Sequences

摘要

Systems and methods for unsupervised representation learning by sorting sequences are provided. An unsupervised representation learning approach is provided which uses videos without semantic labels. The temporal coherence as a supervisory signal can be leveraged by formulating representation learning as a sequence sorting task. A plurality of temporally shuffled frames (i.e., in non-chronological order) can be used as inputs and a convolutional neural network can be trained to sort the shuffled sequences and to facilitate machine learning of features by the convolutional neural network. Features are extracted from all frame pairs and aggregated to predict the correct sequence order. As sorting shuffled image sequence requires an understanding of the statistical temporal structure of images, training with such a proxy task can allow a computer to learn rich and generalizable visual representations from digital images.