Revisiting Document Length Hypotheses: A Comparative Study of Japanese Newspaper and Patent Retrieval

摘要

NTCIR-4 experiments of the CLIR J-J (Japanese monolingual newspaper retrieval) and patent tasks are described, focusing on comparative studies of two test collections and two retrieval approaches in view of document length hypotheses. TF~*IDF outperformed the language modeling approach in the CLIR J-J task whereas the language modeling approach performed better in the patent task. Two different document length hypotheses behind two tasks/collections are assumed by analyzing document length distributions of relevant/retrieved documents in the NTCIR-3 and -4 collections. Given these hypotheses, TF~*IDF is easily adapted to patent retrieval tasks. Document length prior probabilities are applied to the language modeling approach. For the patent task, task-specific techniques, such as IPC priors and different indexing strategies, are evaluated and reported. To facilitate retrieval from large patent collections, a simple distributed search strategy is applied and found to be efficient, despite a slight deterioration of effectiveness. We found that TF~*IDF performed similarly to the language modeling runs against the patent collection by controlling the document length normalization, whereas the language modeling approach does not perform as well as TF~*IDF, despite calibration against the CLIR J-J collection. The different characteristics of the document lengths of the two test collections are illustrated through comparative studies.