Systems and methods for multimodal multilabel tagging of video

摘要

Multimodal multilabel tagging of video content may include labeling the video content with topical tags that are identified based on extracted features from two or more modalities of the video content. The two or more modalities may include (i) a video modality for the object, images, and/or visual elements of the video content, (ii) a text modality for the speech, dialog, and/or text of the video content, and/or (iii) an audio modality for non-speech sounds and/or sound characteristics of the video content. Combinational multimodal multilabel tagging may include combining two or more features from the same or different modality in order to increase the contextual understanding of the features and generate contextually relevant tags. Video content may be labeled with global tags relating to overall topics of the video content, and different sets of local tags relating to topics at different segments of the video content.