基于Hadoop架构的大数据量日志存储和检索优化

摘要

Hadoop is an open-source distributed parallel programming architecture that realized MapReduce computing model. By using Hadoop, programmers can write distributed parallel program easily and complete the calculation of the vast amounts of data by applying it to computer cluster. This article introduces Hadoop system design and implementation, how to improve the execution effciency and speed of HDFS and MapReduce, and the deployment method of Hadoop.%　　Hadoop 是一个实现 MapReduce 计算模型的开源分布式并行编程框架，借助于 Hadoop，程序员可以轻松编写分布式并行程序，并将其运行于计算机集群上，完成海量数据的计算。本文介绍了基于Hadoop架构的系统的设计与实现，介绍如何提高Hadoop的分布式文件管理系统（HDFS）和MapReduce的执行效率与速度，以及 Hadoop 的安装部署和基本运行方法。