A Two-Stream Network with Image-to-Class Deep Metric for Few-Shot Classification

摘要

Few-shot learning in image classification aims to learn classifiers for new classes when few examples are available for each class. Though recent work has greatly advanced promising classification performance, they mainly focus on the feature maps extracted from RGB images and the task-invariant image-to-image metrics. In this paper, we argue that richer features need to be learned and the general metrics are not effective enough due to the scarcity of examples in few-shot learning. Specifically, we propose a Two-Stream Neural Network (TSNN) with a learnable Image-to-Class Deep Metric (ICDM) for few-shot learning, which is trained end-to-end from scratch upon the recent episodic training mechanism. We not only extract features from RGB images to find contrast differences in semantic information, but also leverage the steganalysis features extracted from a steganalysis rich model filter layer to discover the local inconsistencies between different categories. Meanwhile, we extend our model to fine-grained few-shot classification, which is benefit from the proposed novel ICDM. The experimental results on three benchmark datasets show that our approach attains superior performance, with the largest improvement of 6.01% in classification accuracy over related competitive baselines.