Storage Mechanism of Processing Magnanimity Small Files Applying HDFS Architecture

摘要

Pointing at the low efficiency problem of I/O operations of small files using HDFS application architecture, this article proposes a new method, that is, add disk buffer on DataNode to reduce addressing time of reading small files, and thus reduce reading time, and reduce the stress bringed by frequent NameNode access, so as to optimize the reading efficiency of small files. Experiment results show that the design programme is feasible, and adding disk buffer benefits for increasing efficiency of processing small files applying HDFS.