Enhancement of data oriented grid scheduling using dynamic error detection

摘要

Traditional distributed computing systems closely couple data handling and computation. The key features of the first batch scheduler specialized in data placement and data movement is Stork. Stork is especially designed to understand the semantics and characteristics of data placement tasks, which can include data transfer, storage allocation and de-allocation, data removal, metadata registration and replica location. The Stork also has its own drawbacks in detecting the failures resulting from back-end system level problems, like connectivity failure which is technically untraceable by users. Error messages are not logged efficiently, and sometimes are not relevant/useful from users' point-of-view. Our study explores the possibility of efficient error detection and reporting system for such environments. Besides, early error detection and error classification have great importance in organizing data placement jobs. It is necessary to have well defined error detection and error reporting methods to increase the usability and serviceability of existing data transfer protocols and data management systems.