Cross-Media Correlation Analysis with Semi-supervised Graph Regularization

摘要

With the rapid development of multimedia data such as text, image, cross-media retrieval has become increasingly important, because users can retrieve the results with various types of media by submitting a query of any media type. The measure of relevance among different media is a basic problem. Existing methods usually only consider the original media instances (such as images, texts) but ignore their patches. In fact, cross-media patches can emphasize the important parts and improve the precision of cross-media correlation. What's more, existing cross-media retrieval methods often focus on modeling the pairwise correlation with the similarity matrix is a constant matrix, while the similarity matrix which is not a constant matrix can improve the accuracy. In this paper, we propose a novel algorithm for cross-media data, called cross-media correlation analysis with semi-supervised graph regularization (CMCA), which can not only take full advantage of both the media instances and their patches in one graph, but also explore the similarity matrix which can improve the correlation between data. CMCA explores the sparse and semi-supervised regularization for different media types, and integrates them into a unified optimization matter, which increases the performance of the algorithm. Comparing with the current state-of-the-art methods on two datasets (i.e., Wikipedia, XMedia), the comprehensive experimental results demonstrate the effectiveness of our proposed approach.