Scalable Knowledge Discovery Through Grid Workflows

摘要

The goal of this effort was to drastically reduce the human effort required to configure and execute new workflows for data analysis from weeks to minutes by eliminating the need for costly human monitoring and intervention. This involves developing end-to-end data analysis systems to analyze data from many different sources and with many different algorithms and analytical tools. Their approach combined three central ideas: 1) workflows with rich representations of algorithmic requirements and data products, 2) semantic representations to enable automatic generation of complex workflows, and 3) grid computing to manage the high performance of many workflows in distributed cross-organization environments.