How well they retrieve fresh news items: News search engine perspective

摘要

Summary People are nowadays opting news search engines for searching news instead of traditional web search engines as, number of specialized news search services have been developed. So it becomes necessary to evaluate these news search systems and help users to select the best one. Lots of work has been done to measure the traditional effectiveness of web search engines, major work has been done for relevance based evaluation using precision based measures, where topical relevance is often the main selection criteria, but less work has been done to measure the time-sensitive effectiveness of the news search systems where freshness matters. In this paper we used a scheme using mathematical statistics to measure the time-sensitive effectiveness of four news search systems, i.e., how well they retrieve the fresh documents. To our knowledge there is a lack of a good measure that combines both time-independent effectiveness and the relative freshness of news items so our scheme, using top ten results for 100 news queries on four news search engines with the basic idea to pull all the relevant results from the news search systems we want to compare together into a single ranked list based on their recency and analyse the relative positions of these results, will be useful in stuffing this gap.