Measuring and Discovering Correlations in Large Data Sets

摘要

In this paper, a class of statistics named ART (the alternant recursive topology statistics) is proposed to measure the properties of correlation between two variables. A wide range of bi-variable correlations both linear and nonlinear can be evaluated by ART efficiently and equitably even if nothing is known about the specific types of those relationships. ART compensates the disadvantages of Reshef's model in which no polynomial time precise algorithm exists and the "local random" phenomenon can not be identified. As a class of nonparametric exploration statistics, ART is applied for analyzing a dataset of 10 American classical indexes, as a result, lots of bi-variable correlations are discovered.