A Fault Tolerant Task Scheduler supporting Dynamic Task Creation

摘要

In recent years, a number of internet parallel computing projects started and gained popularity quickly. One of the noticeable architectural characteristics in common among these projects is the existence of a centralized task-queue server. Although management is easier under such architecture, the task-queue server unfortunately becomes the single point of failure and communication bottleneck in the system. One can circumvent above problems by server replication and load-balancing among the replicated servers.