C-SWF: A Lightweight Scientific Workflow System for Astronomical Data Processing

摘要

Due to the urgent requirements of mass data processing in the field of astronomy, with the advent of network grid computing technique, scientific workflow technique was presented and quickly adopted for distributed astronomic data processing. However, current existing scientific workflow systems are too complex and enormous in system deploying and system manipulation. Few people can master it in short period. In this paper, we presented a lightweight scientific workflow system, C-SWF, which is specially designed for astronomy. All required fundamental functions, such as task customization, data movement, provenance and task re-run mechanisms, are fully implemented. Comparing with the existing scientific workflow system, C-SWF provides many useful features such as simple, high performance and easy to deploy in order to meet the requirements of astronomers.