A sound-based video clipping framework toward sports scenes

摘要

Video clipping system is very important in many intelligent applications. In order to shorten the time of video and extract the framework of the video, many methods have been proposed. But these methods just considered videos without taking sound into account. As we know, sound is also an important information for image processing. For example, many sport match videos include rich sound of audience and commentator such as NBA. In addition, human pay more attention to some video clips of interest (VCOI) such as scoring time instead of pause. So in this paper, we propose a sound-based video clipping framework toward specific sports scenes. First, we convert sound of sport videos to sonogram. For some aesthetically-pleasing images (APIM) such as slam dunk or jump shot, a set of object patches are selected using BING feature. Then, these object patches are ordered by our active object patches ranking algorithm. After that, ordered object patches and sonogram are fed into CNN respectively to obtain patch-level deep feature. In order to obtain image-level deep representation, deep feature extracted from ordered object patches are aggregated statistically into a deep representation. Finally, probabilistic model is used to select VCOI and APIM. Experiments on some NBA basketball matches have shown the effectiveness of our video clipping framework.