System and method for executing map-reduce tasks in a storage device

摘要

A system and method of providing enhanced data processing and analysis in an infrastructure for distributed computing and large-scale data processing. This infrastructure uses the Hadoop™ framework to divide an application into a large number of small fragments of work, each of which may be performed on one of a large number of compute nodes. The work may involve map tasks and reduce tasks which may be used to categorize and analyze large amounts of data in distributed systems. This infrastructure includes a cluster with a master node and a plurality of slave nodes. The slave nodes may include, or may be, intelligent solid-state drives capable of executing Map-Reduce functions. The use of intelligent solid-state drives reduces the need to exchange data with a CPU in a server.