Intelligent Multimedia Annotation and Interaction Using Semantic Musical Features: Encoding Human-Centric Music Intelligence for Musically Plausible Human-Media Interactions

摘要

Semantic musical features reflect in-depth understanding of the music, instead of the uninterpreted music content, and serve as idea choices for multimedia content annotations. The proposed semantic music features are based on human music interpretations and their computational implementations. When employed for multimedia applications, these features enable us to simulate human-music interactions. This musical relevance provides significant performance improvement over conventional score or audio based multimedia annotation systems. Two types of semantic musical features, including reductive music analysis and musical expressive features, are introduced. The details of their feature extraction algorithms and semantic interpretations are also illustrated.