We examine multi-modal information retrieval from broadcast video where text can be read on the screen through OCR and speech recognition can be performed on the audio track. OCR and speech recognition are compared on the 2001 TREC Video Retrieval evaluation corpus. Results show that OCR is more important that speech recognition for video retrieval. OCR retrieval can further improve through dictionary-based post-processing. We demonstrate how to utilize imperfect multi-modal metadata results to benefit multi-modal information retrieval.
展开▼
机译：我们研究了从广播视频中检索多模式信息的方法，其中可以通过OCR在屏幕上读取文本，并且可以在音轨上执行语音识别。 OCR和语音识别在2001 TREC Video Retrieval评估语料库中进行了比较。结果表明，OCR比语音识别对视频检索更重要。通过基于字典的后处理，OCR检索可以进一步改善。我们演示了如何利用不完善的多模式元数据结果来受益于多模式信息检索。
展开▼
