Workload-aware distributed data processing apparatus and method for processing large data based on hardware acceleration

摘要

A workload-aware distributed data processing apparatus and method for processing large data based on hardware acceleration are provided. The data processing apparatus includes a memory buffer including partitions. The data processing apparatus further includes a partition unit configured to distribute a mapping result to the partitions based on a partition proportion scheme. The data processing apparatus further includes a reduce node configured to receive content of a corresponding one of the partitions, and perform a reduction operation on the content to generate a reduce result.