基于HADOOP分布式系统的数据处理分析

摘要

With the rapid growth of information and data, cloud computing technology provides a fairly good solution for massive data processing. Hadoop, as an open source implementation of distributed parallel programming framework in cloud computing technology, could effeciently complete the processing of massive data by parallel computing and storage capacity of clusters. Cloud computing could create more value for the society. Finally with cheap computer resources a distributed cloud computing platform could be built up by Hadoop, and this has the superiorities of high fault-tolerance, high efficiency, high scalability. Meanwhile the performance test and analysis of data processing on Hadoop cluster is completed.%各种信息数据在迅猛增长，云计算技术的出现为海量数据的处理提供了良好的解决方案。Hadoop 是云计算技术中分布式并行编程框架的开源实现，它利用集群的并行计算和存储能力，高效的完成对海量数据的处理，云计算能够为社会创造出更大的价值。最后通过Hadoop组织廉价的计算机资源搭建了一个分布式云计算平台，具有高容错、高效性、高扩展性等优势，并完成对数据处理能力和平台可扩展性进行性能测试与分析。