Flexible and Generic Workflow Management

摘要

Running and combining HPC applications often leads to complex scientific workflows, even more when code is to be executed in several different computing platforms. We present a flexible and platform independent framework for workflow definition and execution based on a redesigned version of the benchmarking environment JUBE. By means of a generalised configuration method this new version of JUBE can now be applied to more complex production, development and testing scenarios. It provides user-defined parameter substitution at all workflow stages, automated job submission, extensive directory and result handling and customisable analysis steps. In this report we demonstrate how it can be used to implement a given workflow representation and how it relates to and differs from other generic workflow management systems.