Incorporating Feature Selection and Clustering Approaches for High-Dimensional Data Reduction

摘要

Data reduction is an important research topic for analyzing mass data efficiently and effectively in the era of big data. The task of dimension reduction is usually accomplished by technologies of feature selection, feature clustering or algebraic transformation. A novel approach for reducing high-dimensional data is initiated in this paper. The main idea of the proposed scheme is to incorporate data clustering and feature selection to transform high-dimensional data into lower dimensions. The incremental clustering algorithm in the scheme is used to handle the number of dimensions, and the relative discriminant variable is design for selecting significant features. Finally, a simple inner product operation is applied to transform original highdimensional data into a low one. Evaluations are conducted by testing the reduction approach on the problem of document categorization. The experimental results show that the reduced data have high classification accuracy for most of datasets. For some special datasets, the reduced data can get higher classification accuracy in comparison with original data.