A Graph-Based Method for Detecting Rare Events: Identifying Pathologic Cells

摘要

Detection of outliers and anomalous behavior is a well-known problem in the data mining and statistics fields. Although the problem of identifying single outliers has been extensively studied in the literature, little effort has been devoted to detecting small groups of outliers that are similar to each other but markedly different from the entire population. Many real-world scenarios have small groups of outliers--for example, a group of students who excel in a classroom or a group of spammers in an online social network. In this article, the authors propose a novel method to solve this challenging problem that lies at the frontiers of outlier detection and clustering of similar groups. The method transforms a multidimensional dataset into a graph, applies a network metric to detect clusters, and renders a representation for visual assessment to find rare events. The authors tested the proposed method to detect pathologic cells in the biomedical science domain. The results are promising and confirm the available ground truth provided by the domain experts.