Investigating Hadoop Architecture and Fault Tolerance in Map-Reduce

摘要

Map-Reduce is often used in implementation of critical and important tasks such as analysis of the scientific data. However, evidences in the past indicate the presence of optional errors that can destroy the results of Map-Reduce. Of course, run times of Map-Reduce like Hadoop can tolerate crash errors, but do not tolerate arbitrary or Byzantine errors. Hence in this paper, at first, the Hadoop architecture in distributed system will be investigated and then Hadoop will be compared with Map-Reduce and finally the Map-Reduce fault tolerance will be investigated.