Implementation of Multilevel Indexing in Search Engines

摘要

Indexing in search engines has been an active area of current researches. Granting efficient and fast access to the index is a key issue for performance of Web Search Engines. Search Engines use inverted indexes that consist of an array of posting lists where each posting list is associated with a term and contains the identifiers of the documents containing the term. A global single large size index takes a long time to search the relevant documents in response to the user queries. This paper proposes multilevel indexing algorithm i.e. creating the indexes at the multiple levels so as to reduce the search time by directing the search to a specific path from higher level indexes to the lower level indexes. The multilevel indexes are created after the hierarchical clusters of the documents have been formed on the basis of document similarity. At each level of documents clusters hierarchy, a separate index is created and the search for a document proceeds from the higher level indexes to the lowest level index which is the document level index and is stored as inverted file. Thus the paper implements the proposed hierarchical algorithm at different levels which optimizes the search process by directing the search to a specific path from higher levels of clustering to the lower levels i.e. from super clusters to mega clusters, then to clusters and finally to the individual documents so that the user gets the best possible matching results in minimum possible time. The paper further presents the graphs showing clusters created at different levels.