User-oriented modelling of scientific workflows for high frequency event data analysis

摘要

Whether it is research scientists in computational physics, astronomy, genomics or financial services, all these varying disciplines have been challenged by the analysis of Big Data. They are all required to perform multi-step analysis tasks to turn this data into actionable insight, from which critical decisions can be made. Two data processing models that have rapidly evolved in the past decade to support data analysts are Complex Event Processing and Scientific Workflows. Our research proposes a hybrid approach, which extends scientific workflows, to incorporate the handling of event-streams. This model not only aims to provide a more efficient approach to analysing high frequency event streams, but also facilitates conceptual modelling of processes - to enable domain experts to build abstract, exploratory analysis processes in a user-friendly manner without the concerns of underlying technology, and transparently maps them to concrete solutions at run-time.