DISTRIBUTED PROCESSING APPARATUS AND METHOD FOR BIG DATA USING HARDWARE ACCELERATION BASED ON WORK LOAD

摘要

To a distributed processing apparatus for large-scale data based on hardware acceleration considering a workload. A data distribution processing apparatus according to an exemplary embodiment includes at least one partitioned memory buffer corresponding to one or more refinement nodes, and a partitioning unit for distributing a mapping result input to each partition based on a distribution ratio plan of each partition. And one or more refinement nodes for receiving and refining the content of the corresponding partitions of the node and the memory buffer to generate a final result. According to the present embodiment, the performance of the data distribution processing can be greatly improved by adaptively distributing the jobs in consideration of the resource conditions such as the processing speed of the job nodes providing the final distributed processing result.