Residual visualization-guided explainable copy-relationship learning for image copy detection in social networks

摘要

In modern times, people are used to sharing individual works, giving opinions, and expressing emotions and sentiments by transmitting a variety of images such as personal photographs, pictures of purchased products, and emoticons on social networks. However, this results in a lot of illegal image copies being distributed, which causes the issues of misleading the public and copyright infringement. To identify such images, recent copy detection approaches learn similarity functions by directly exploring neural networks such as Siamese and pseudo-Siamese networks and thereby determine the copy relationships between images. Unfortunately, since such approaches directly use these networks as black boxes, without considering the essential differences between image copies and similar images, handling the prevalent need for copy detection, i.e., identifying image copies from similar images, remains a challenge. To address the above issue, first, a residual visualization scheme is proposed in this study to visualize the residual image by subtracting the original image from the geometrically aligned test image, thereby revealing the relationship between the two images. Then, guided by the observations on the residual visualization results, a newly designed explainable copy-relationship learning network (ECRLN) architecture is presented. In this network architecture, the characteristics of residual images, ranging from local details to global appearances and from low-level visual patterns to high-level semantics, are sufficiently utilized to learn a measurement function for computing the copy-relationship confidence between images. The extensive experiments conducted on the publicly available datasets as well as our own challenging dataset demonstrate that the proposed approach can accurately identify image copies from similar images with good interpretability and achieve state-of-the-art performance in detection accuracy and training efficiency. (C) 2021 Elsevier B.V. All rights reserved.