Dual-Y network: infrared-visible image patches matching via semi-supervised transfer learning

摘要

Infrared-visible image patches matching has many applications, such as target recognition, vision-based navigation, and others. At present, deep learning has achieved excellent performance in visible image patches matching. Due to imaging differences in infrared-visible images and the small number of supervised samples, the existing networks cannot solve them well. We propose a Dual-Y network based on semi-supervised transfer learning to address the challenges. Since the infrared and visible image patches have similarities in the same scene, adversarial domain adaptation uses visible images as the source domain and infrared images as the target domain to solve different imaging principles. Through a large number of unannotated samples training, the two convolutional autoencoders in our network can reconstruct the infrared-visible image patches respectively to improve feature representation ability. In the high-level convolution layers, the cross-domain features are extracted by sharing weights in two domains. The adversarial domain adaptation achieves the domain confusion in the high layers. In the low-level convolutional layers, independent weights in two branches can ensure that the domain-related imaging principles can be preserved. Finally, the annotated data are used for supervised training in infrared-visible image patches matching. With the improvements above, the matching accuracy increase by 10.78%, compared with the fine-tuning based on the pre-training module.