Gini diversity index, Hamming distance, and curse of dimensionality

摘要

The celebrated Gini(-Simpson) biodiversity index has found very useful applications in ecology, bio-environmetrics, econometry, psychometry, genetics, and lately in bioinformatics as well. In econommic, social and environmental sciences, there are some ordinal categorical variables or grouped data sets, so that it needs an alternative means of data summarization and interpretation. However, in many other applications in genetics and bioinformatics, mostly, categorical data models, without possibly an ordering of the categories, crop up, which may preempt routine use of conventional measures of quantitative diversity analysis. Further, in real life problems, mostly, genuine and (often enormously large) multidimensional data models, are encountered. The Hamming distance incorporates and generalizes the concept of the Gini-Simpson diversity index in a variety of multidimensional setups, without making very stringent structural regularity assumptions. In bioinformatics as well as many other large biological system analyses, the curse of dimensionality is a genuine concern. The role of Hamming distance based analysis is appraised in this perspective. Subgroup or MANOVA decomposability aspects are specially appraised in this setup.