How Large Is theWorldWideWeb?

摘要

The problem of assessing the size of the World Wide Web is extremely difficult becausernsampling directly from theWeb is not possible. Several groups of researchers have investedrnconsiderable effort to develop sound sampling schemes which involve submitting a numberrnof queries to several major search engines. In this paper we present a statistical approach forrnthe analysis of datasets collected by query-based sampling, utilizing a hierarchical Bayesrnformulation of the Rasch model for multiple list population estimation. We show thatrnour procedures accord with the real-world constraints and consequently they let us makerncredible inferences about the size of the World Wide Web.