Efficient All Relevant Feature Selection with Random Ferns

摘要

Many machine learning methods can produce variable importance scores expressing the usability of each feature in context of the produced model; those scores on their own are yet not sufficient to generate feature selection, especially when an all relevant selection is required. There are wrapper methods aiming to solve this problem, mostly focused around estimating the expected distribution of irrelevant feature importance. However, such estimation often requires a substantial computational effort. In this paper I propose a method of incorporating such estimation within the training process of a random ferns classifier and evaluate it as an all relevant feature selector, both directly and as a part of a dedicated wrapper approach. The obtained results prove its effectiveness and computational efficiency.