基于聚类的动态负载均衡在数据采集上的应用

摘要

为了构建一个基于微博的社会网络,需要提供大量的微博数据源,那么如何才能实时高效的获取微博信息是构建微博社会网络面临的重大挑战。本文提出了一种基于聚类的动态负载均衡数据采集方法,将聚类算法与动态负载均衡结合是一次新的尝试,测试表明,能够满足对微博数据采集的需求。%To build social networks based on microblog,we need to provide large microblogging data source,however how to efficient access to the microblogging information is to build social networks of the major challenges facing.this paper presents dynamic load balancing method of data collection based on clustering,combined with clustering and dynamic load balancing is a new attempt,the tests show that it can meet the demand for microblogging data collection.