Combining clustering of variables and feature selection using random forests

摘要

High-dimensional data classification is a challenging problem. A standardapproach to tackle this problem is to perform variables selection, e.g. usingstep-wise or LASSO procedures. Another standard way is to perform dimensionreduction, e.g. by Principal Component Analysis or Partial Least Squareprocedures. The approach proposed in this paper combines both dimensionreduction and variables selection. First, a procedure of clustering ofvariables is used to built groups of correlated variables in order to reducethe redundancy of information. This dimension reduction step relies on the Rpackage ClustOfVar which can deal with both numerical and categoricalvariables. Secondly, the most relevant synthetic variables (which are numericalvariables summarizing the groups obtained in the first step) are selected witha procedure of variable selection using random forests, implemented in the Rpackage VSURF. Numerical performances of the proposed methodology calledCoV/VSURF are compared with direct applications of VSURF or random forests onthe original $p$ variables. Improvements obtained with the CoV/VSURF procedureare illustrated on two simulated mixed datasets (cases $nextgreater{}p$ and$nextless{}extless{}p$) and on a real proteomic dataset.