Automatic Labanotation Generation, Semi-automatic Semantic Annotation and Retrieval of Recorded Videos

摘要

Over the last decade, the volume of unannotated user-generated web content has skyrocketed but manually annotating data is costly in terms of time and resources. We leverage the advancements in Machine Learning to reduce these costs. We create a semantically searchable dance database with automatic annotation and retrieval. We use a pose estimation module to retrieve body pose and generate Labanotation over recorded videos. Though generic, it provides an essential application due to large amount of videos available online. Labanotation can be further exploited to generate ontology and is also very relevant for preservation and digitization of such resources. We also propose a semiautomatic annotation model which generates semantic annotations over any video archive using only 2-4 manually annotated clips. We experiment on two publicly available ballet datasets. High-level concepts such as ballet pose and steps are used to make the semantic library. These also act as descriptive meta-tags making the videos retrievable using a semantic text or video query.