Sentiment key frame extraction in user-generated micro-videos via low-rank and sparse representation

摘要

It is a prevailing trend that the user-generated content (UGC) on social media is shifting toward mobile video and micro content. Sentiment analysis and emotion recognition extract the opinions expressed in UGC and are important to understand the fast-growing mobile micro-videos. Although extensive research efforts have been devoted to it, most of the existing studies pre-processed the video before extracting sentiment features, pre-sampling as an example. Due to the rather unstructured nature of user-generated videos and the sparsely expressed emotions, sampling before sentiment analysis can remove visual contents important to understand the video. In this paper, a novel unsupervised sentiment key frame extraction (SKFE) model based on low-rank and sparse representation is proposed. Sparsity is the important characteristic distinct to a video frame. The low-rank constraint is helpful to improve the robustness to noise and outliers. The 1 2 , 1 -norm sparsity is adopted to improve the robustness via different indexes running through the feature dimensions and among the video frames. To better characterize the relationship between the local features and alleviate the sensitiveness of sparse coding, a regularization term based on the Laplacian matrix is introduced to preserve the consistency of sparse codes for similar local features. The experimental results on publicly available datasets demonstrate the effectiveness of the proposed SKFE model. (C) 2020 Elsevier B.V. All rights reserved.