A Novel Technique for Web Pages Clustering Using LSA and K-Medoids Algorithm

摘要

The extensibility of various web documents available on the web made a critical challenge for many serious tasks such as information retrieval (IR), content monitoring, and indexing. Web documents could be any type of data that can be requested by user and delivered from web server through several web browsers. Most of web documents contain textual contents and are typically called web pages. However, in order to perceive and discover knowledge from these pages, novel techniques are required that have been never applied in other domains. In this paper, a new approach has been proposed by performed latent semantic analysis (LSA) on the result of VSM, which involves the correlation among web pages to their extracted features. The result of LSA involves the matrices that reflect the correlation between the web pages to their related concepts, which were used frequently for retrieving process. PAM (K-Medoids) algorithm was used with respect to semantic space, to portion the web pages into coherent groups. One of the most challenges in any clustering algorithm is to identify the correct number of clusters for the given data. Hence, two approaches are used for this manner: Elbow graph analysis to estimate the number of cluster range based on (SSE) values and clustering evaluation metrics. Calinski-Harabasz criterion (CH) and Silhouette Coefficient (SC) are the best well-known evaluation metrics commonly used in partitioning-based algorithms. UOT has been considered to evaluate the proposed system, and the results are shown in the proposed system to achieve high accuracy results to separate the similar pages into coherent groups.