Parallel methods for the update of partitioned inverted files

摘要

Purpose - An issue that tends to be ignored in information retrieval is the issue of updating invertedrnfiles. This is largely because inverted files were devised to provide fast query service, and much workrnhas been done with the emphasis strongly on queries. This paper aims to study the effect of usingrnparallel methods for the update of inverted files in order to reduce costs, by looking at two types ofrnpartitioning for inverted files: document identifier and term identifier.rnDesign/methodology/approach - Raw update service and update with query service are studiedrnwith these partitioning schemes using an incremental update strategy. The paper uses standardrnmeasures used in parallel computing such as speedup to examine the computing results and also therncosts of reorganising indexes while servicing transactions.rnFindings - Empirical results show that for both transaction processing and index reorganisation therndocument identifier method is superior. However, there is evidence that the term identifier partitioningrnmethod could be useful in a concurrent transaction processing context.rnPractical implications - There is an increasing need to service updates, which is now becoming arnrequirement of inverted files (for dynamic collections such as the web), demonstrating that a shift inrnrequirements of inverted file maintenance is needed from the past.rnOriginality/value - The paper is of value to database administrators who manage large-scale andrndynamic text collections, and who need to use parallel computing to implement their text retrievalrnservices.