Workflow Description Language for defining Big Earth Data processing tasks

摘要

In this paper we are presenting an approach for the flexible description of Earth Observation (EO) data processing tasks with varying degrees of complexity. This approach is based on a simple and compact description language which offers the user the possibility of creating complex processing tasks by representing them in the form of a network or directed graph consisting of independent functional units, called operators. This kind of description allows for the partitioning of the processing tasks into smaller, more manageable processing units which can potentially be run in parallel. By relying on collections of readily available functional units to be employed in the creation of processing tasks, our system attempts to maximize the component reuse rate, while at the same time taking advantage of the design's modular nature in order to exploit opportunities for task parallelism. To make all of this possible, we envisaged the use of the specially-designed description language, called WorDeL, which we will be describing in detail throughout this paper.