Jobcast - Parallel and Distributed Processing Framework: Data Processing on a Cloud Style KVS Database

摘要

In this paper, we propose a new architecture for parallel and distributed processing framework, gJobcasth, which enables data processing on a cloud style KVS database. Nowadays, lots of KVS (as known as Key Value Store) systems exist which achieve high scalability for data spaces among a huge number of computers. Some of KVS implementations use gconsistent hashh algorithm to identify the backend data node to store a pair of key and value. Jobcast also uses consistent hash algorithm for a distribution strategy and has a capability to store key and value pairs into huge number of computers as a KVS system. Furthermore, Jobcast also distributes gjobsh into data nodes for parallel and distributed processing. In this paper, we introduce a basic architecture of Jobcast and evaluate a data processing performance for a typical example.