Exploratory data analysis using set operations and ordinal mapping.

摘要

Exploratory data analysis requires the ability to issue ad hoc queries to filter and summarise data sets. As the sizes of health data sets grow, traditional methods of processing data have difficulty in providing acceptable response times for such queries. An alternative method is described which combines complete vertical partitioning of data with set operations on ordinal mappings (SOOM). An initial implementation of the technique provides significantly better performance than a conventional SQL database on typical exploratory data analysis queries. The use of parallel, distributed computation to further increase the performance of the technique appears to be feasible.