Random Forest Regression Based on Partial Least Squares: Connect Partial Least Squares and Random Forest

摘要

Partial Least Squares (PLS) Regression is lack of theoretical guidance of rules to achieve the nonlinear by the quasi linearization rule, and its accuracy declines in the face of the unknown variables distribution. Furthermore, the loss of information is easy to arise for the mean processing of the leaf in the Regression Tree of the traditional Random Forest Regression. On this basis, Partial Model Tree (PMT) is proposed combining Partial Least Squares Regression with Regression Tree, to achieve the nonlinear regression by constructing multiple linear fragments of Partial Least Squares to complete linear approximation of the unknown variables, and the information loss issue caused by that the leaf nodes are treated by direct mean processing is avoided, when PLS regression is used in the leaf nodes. It applies PMT to ensemble learning to build Partial Least Squares of Random Forests Regression (PLS-RFR), improving the generalization ability of PMT. The ability of explanation and predicting get improved in the experiment data of MaXingShiGan decoction of the monarch drug to treat the asthma or cough and five sample sets in the UCI Machine Learning Repository. Finally, it verifies that the PMT and RFPLS possess a certain degree of validity and correctness.