A scalable GPU-enabled framework for training deep neural networks

摘要

In the last fifteen years, Big Data created a new generation of data analysis problems, which does not only involve the problems themselves but also the way these data are handled. Since managing terabytes of data without a proper infrastructure is unfeasible, a smart way to process these data is also necessary. A solution to this aspect deals with the creation of general algorithms that learn from observations. In this context, Deep Learning promises general, powerful, and fast machine learning algorithms, moving them one step closer to artificial intelligence. Nevertheless, fitting a deep learning model may require an huge amount of time, thus, the need of scalable infrastructures for processing large scale data sets has become ever more meaningful. In this paper, we present a framework for training these deep neural networks using heterogeneous computing resources of either grid or cloud infrastructures. The framework lets the end-users define the deep architecture they need for processing their own Big Data, while dealing with the execution of the learning algorithms on a distributed set of nodes (through Apache Flink) as well as with offloading the computation on multiple Graphics Processing Units.