Generalized Information-Theoretic Measures for Feature Selection

摘要

Information-theoretic measures are frequently employed to select the most relevant subset of features from datasets. This paper focuses on the analysis of continuous-valued features. We compare the common approach with discretization of features prior the analysis, to the direct usage of exact values. Due to the overwhelming costs of computing continuous information-theoretic measures based on Shannon entropy the Renyi and Tsallis generalized measures are considered. To enable computation with continuous Tsallis measures a novel modification of the information potential is introduced. The quality of the analysed measures was assessed indirectly through the classification accuracy in conjuction with the greedy feature selection process. The experiments on datasets from UCI repository show considerable improvements of the results when using both generalized continuous measures.