JS4Cloud: script-based workflow programming for scalable data analysis on cloud platforms

摘要

Workflows are an effective paradigm to model complex data analysis processes, such as knowledgerndiscovery in databases applications, which can be efficiently executed on distributed computing systemsrnsuch as a Cloud platform. Data analysis workflows can be designed through visual programming, whichrnis a convenient design approach for high-level users. On the other hand, script-based workflows are a usefulrnalternative to visual workflows, because they allow expert users to program complex applications morerneffectively. In order to provide Cloud users with an effective script-based data analysis workflow formalism,rnwe designed the JS4Cloud language. The main benefits of JS4Cloud are as follows: (i) it extends thernwell-known JavaScript language while using only its basic functions (arrays, functions, and loops); (ii) itrnimplements both a data-driven task parallelism that automatically spawns ready-to-run tasks to the Cloudrnresources and data parallelism through an array-based formalism; and (iii) these two types of parallelism arernexploited implicitly so that workflows can be programmed in a fully sequential way, which frees users fromrnduties like work partitioning, synchronization, and communication. We describe how JS4Cloud has beenrnintegrated within the data mining cloud framework (DMCF), a system supporting the scalable execution ofrndata analysis workflows on Cloud platforms. In particular, we describe how data analysis workflows modeledrnas JS4Cloud scripts are processed by DMCF by exploiting parallelism to enable their scalable executionrnon Clouds. Finally, we present some data analysis workflows developed with JS4Cloud and the performancernresults obtained by executing such workflows on DMCF.