Study and analysis of hadoop cluster optimization based on configuration properties

摘要

Hadoop framework consists of multiple components. One set of configuration parameters may not be suitable for all types of applications. Hence, for optimum performance, each framework parameter needs to be carefully chosen and adjusted. This paper highlights some of the important configuration properties related to Block size, memory allocation, CPU allocation, Number of MapReduce jobs, Job scheduling and JVM which should be tuned for improved Hadoop application performance. The result of some Benchmark problems are analyzed.