Passage Retrieval vs. Document Retrieval for Factoid Question Answering

摘要

QA systems are usually evaluated end-to-end, on the basis of their ability to find answers to questions. Unfortunately, QA systems are complex ― with many subsystems ― and an end-to-end evaluation often does not allow us to understand the relative contribution that each subsystem makes to the overall performance. Here, we examine the IR subsystem in isolation. Our results suggest that full-document retrieval may be desirable in a QA system, since higher precision may translate into better performance. However, when full documents are retrieved, post-retrieval passage selection may still be required. The MultiText method, even when used for full-document retrieval, identifies a hotspot where the answer is more likely to found, combining the advantages of both passage and document retrieval.