The Constructing of Test Set on Chinese Information Retrieval

摘要

In this paper,we propose a Template-Matching strategy combined with statistics in the tasks of human evaluation of NTCIR to avoid the arbitrariness of human evaluators.By practical operation and within-system checkup,the strategy helps avoid the arbitrariness of human evaluators and provides a benchmark for human evaluation of NTCIR.