Known-item Search (KIS) in video: Survey, experience and trend

摘要

This paper provides a survey on the notable performers submitted to TRECVid 2010, under Known-item Search (KIS) task. It also gives an insight as well as the lessons learnt discovered by the top ranked system. Most systems used multi-modal features include: low level feature (color and SIFT feature), high level feature (HLF - of which 130 concepts have been released by participants taking part under HLF task), and the metadata given as well as ASR from the audio track of each video. As for the video search approaches, machine learning such as Support Vector Machine (SVM) was employed such as the work done by Dublin City University (DCU). The work reported by the National University of Singapore (NUS), used a slightly different method for video search process. Upon receiving a query from the user, the system submitted the query to YouTube to get initial result. Tag and comments of these videos are then collected. Among these, the top performer employed query to modality mapping approach, of which each query is segmented into sub-queries (classes of visual-cue, audio-cue, and main-concept), each of which will be handled by different detectors. The method achieved the best performing system under this task with a mean inverted rank of 0.454 for automatic search and 0.727 for interactive search. The system is able to scale to handle online and real-time search with cloud environment.