Research on Deep Web Query Interface Clustering Based on Hadoop

摘要

How to cluster different query interfaceseffectively is one of the most core issues when generatingintegrated query interface on Deep Web integration domain.However, with the rapid development of Internet technology,the number of Deep Web query interface shows an explosivegrowth trend. For this reason, the traditional stand-aloneDeep Web query interface clustering approaches encounterbottlenecks in terms of time complexity and spacecomplexity. After further study of the Hadoop distributedplatforms and Map Reduce programming model, a DeepWeb query interface clustering algorithm based on Hadoopplatform is designed and implemented, in which the VectorSpace Model (VSM) and Latent Semantic Analysis (LSA)are employed to represent “Query Interfaces-Attributes”relationships. The experimental results show that theproposed algorithm has better scalability and speedup ratioby using Hadoop architecture.