Design and implementation of DeepDSL: A DSL for deep learning

摘要

Deep Learning(DL) has found great success in well-diversified areas such as machine vision, speech recognition, and multimedia understanding. However, the state-of-the-art tools (e.g. Caffe, TensorFlow, and CNTK), are programming libraries with many dependencies and implemented in languages such as C++ that need to be compiled to a specific runtime environment and require users to install the entire tool libraries for training or inference, which limits the portability of DL applications.In this work, we introduceDeepDSL, a domain specific language (DSL) embedded in Scala, that compiles DL networks encoded with DeepDSL to efficient, compact, and portable Java source programs for DL training and inference. DeepDSL represents DL networks as abstract tensor functions, performs symbolic gradient derivations to generate Intermediate Representation (IR), optimizes the IR expressions, and translates the optimized IR expressions to Java code that runs on GPU without additional dependencies other than the necessary GPU libraries and the related invocation interfaces: a small set ofJNI(Java Native Interface) wrappers. Our experiments show DeepDSL outperforms existing tools in several benchmark programs adopted from the current mainstream Deep Neural Networks (DNNs).