Visualizing large data volumes utilizing initial sampling and multi-stage calculations

摘要

Embodiments visualize large data volumes utilizing initial sampling to reduce size of a dataset. This sampling may be random in nature. The sampled dataset may be refined (wrangled) by binning, grouping, cleansing, and/or other techniques to produce a wrangled sample dataset. A user defines useful end visualization(s) by inputting expected dimension/measures. From these visualizations of sampled data, minimal grouping sets are deduced for application to the full dataset. The user publishes/schedules the wrangled operation and grouping sets definition. Based on this, a wrangled dataset and grouping sets are produced in the big data layer. When the user accesses the visualization(s), minimal grouping sets are retrieved in the in-memory engine of the client and processed by an in-memory database engine according to the common processing plan. This produces result sets and a final set of visualizations of the full dataset, in which the user can recognize valuable data trends and/or relationships.