Prediction of Compound Profiling Matrices Part II:Relative Performance of Multitask Deep Learning and Random ForestClassification on the Basis of Varying Amounts of Training Data

摘要

Currently, there is a high level of interest in deep learning and multitask learning in many scientific fields including the life sciences and chemistry. Herein, we investigate the performance of multitask deep neural networks (MT-DNNs) compared to random forest (RF) classification, a standard method in machine learning, in predicting compound profiling experiments. Predictions were carried out on a large profiling matrix extracted from biological screening data. For model building, submatrices with varying data density of 5–100% were generated to investigate the influence of data sparseness on prediction performance. MT-DNN models were directly compared to RF models, and control calculations were also carried out using single-task DNNs (ST-DNNs). On the basis of compound recall, the performance of ST-DNN was consistently lower than that of the other methods. Compared to RF, MT-DNN models only yielded better prediction performance for individual assays in the profiling matrix when training data were very sparse. However, when the matrix density increased to at least 25–45%, per-assayRF models met or partly exceeded the prediction performance of MT-DNNmodels. When the average performances of RF and MT-DNN over the gridof all targets were compared, MT-DNN was slightly superior to RF,which was a likely consequence of multitask learning. Overall, therewas no consistent advantage of MT-DNN over standard RF classificationin predicting the results of compound profiling assays under varyingconditions. In the presence of very sparse training data, predictionperformance was limited. Under these challenging conditions, MT-DNNwas the preferred approach. When more training data became availableand prediction performance increased, RF performance was not inferiorto MT-DNN.