Finding User Clusters in Sina Microblog

摘要

Sina microblog has been a very popular social microblog service in recent years. However it's difficult to analyze the network structure of Sina microblog because of the huge amount of users. The emergence of cloud computing gives us a new approach to analyze large-scale social networks. Hadoop is a widely used cloud computing platform, several clustering algorithms such as K-means and Canopy have already been implemented on it. However, the initial cluster centers of K-means are hard to select. Canopy provides a way to choose initial centers, but it is not suitable for very large data sets, and both traditional K-means and Canopy K-means converge very slowly. This paper proposes an improved method to cluster microblog users based on their relationship. We name our method "Weight Partitioned Canopy K-means" (WPCK), implement it on Hadoop cluster, and test it along with existing methods. Experimental results show that WPCK can reduce the number of iterations by about 1/3 of traditional K-means and Canopy K-means, while their performance are almost the same.