Dynamically Generated Metadata and Replanning by Interleaving Workflow Generation and Execution

摘要

Workflow engines typically plan an entire workflow and then submit it for execution, and have limited replanning capabilities when the workflow execution fails. This paper presents an approach for interleaving planning and execution. The approach supports the incremental submission of partial workflows for execution until completion. As new metadata is generated dynamically during execution for all new data products, the workflow system can incorporate that dynamically generated metadata in the workflow planning process. The approach also supports replanning in case a resource is no longer available and in case of failure, not just by reassigning resources but also by redesigning the plan by replacing components that may fail to execute. The aproach is implemented and integrated with the WINGS workflow system, and is being used for a medical application.