JEREMIE: Joint Semantic Feature Learning via Multi-relational Matrix Completion

摘要

The relations among heterogeneous data objects (e.g., image, tag, user and geographical point-of-interest (POI)) on interactive Online Social Media (OSM) play an important information source in describing complicated connections among Web entities (users and POIs) and items (images). Jointly predicting multiple relations instead of single relation completion in separate tasks facilitates sufficient knowledge sharing among heterogeneous relations and mitigate the information imbalance among different tasks. In this paper, we propose JEREMIE, a Joint SEmantic FeatuRe LEarning model via Multi-relational Matrix ComplEtion, which jointly complements the semantic features of different entities from heterogeneous domains. Specifically, to perform appropriate information averaging, we first divide the social image collection into data blocks according to the affiliated user and POI information, where POIs are detected by mean shift from the GPS information. Then we develop a block-wise batch learning method which jointly learns the semantic features (e.g., image-tag, POI-tag and user-tag relations) by optimizing a transductive matrix completion framework with structure preservation and appropriate information averaging functionality. Experimental results on automatic image annotation, image-based user retrieval and image-based POI retrieval demonstrate that our approach achieves promising performance in various relation prediction tasks on six city-scale OSM datasets.