Overview of the INEX 2010 focused relevanceudfeedback track

摘要

The INEX 2010 Focused Relevance Feedback track offereduda refined approach to the evaluation of Focused Relevance Feedback algorithms through simulated exhaustive user feedback. As in traditional approaches we simulated a user-in-the loop by re-using the assessments of ad-hoc retrieval obtained from real users who assess focused ad-hocudretrieval submissions. The evaluation was extended in several ways: the use of exhaustive relevance feedback over entire runs; the evaluation of focused retrieval whereudboth the retrieval results and the feedback are focused; the evaluation was performed over a closed set of documents and complete focused assessments; the evaluation was performed over executable implementations of relevance feedback algorithms; and �finally, the entire evaluationudplatform is reusable. We present the evaluation methodology, its implementation, and experimental results obtained for nine submissions from three participatingudorganisations.