System and method for utilizing computational power of a server farm

摘要

Disclosed are systems and methods for parallel processing an input data set. During a map stage of a computation, starting with a first virtual machine (VM) acting as a parent VM and an input data set, the system clones the parent VM to generate at least one linked clone child VM. The system further divides the input data set into a first chunk for the parent VM and a second chunk for the at least one child VM by determining a starting pointer for each chunk. Each chunk is processed by a VM to generate an intermediate data result, which is stored in a network storage device. The plurality of VMs then perform a reduce stage on the plurality of intermediate data results stored in the network storage device.