OPTIMIZED FULL-SPECTRUM ORDER STATISTICS-BASED CARDINALITY ESTIMATION

摘要

Systems and methods are disclosed for optimizing full-spectrum cardinality approximations on big data utilizing an optimized order statistics technique. To accomplish the foregoing, a multiset of objects that each corresponds to one of a plurality of objects associated with a resource are obtained. A compound data object is populated at least in part with data that is derived based on generated decimal fraction hash values that correspond to each object in the obtained multiset. The populated compound data object is processed with a full-spectrum arithmetic mean estimation operation that can accurately determine a cardinality estimate for the obtained multiset using less resources and time when compared to traditional techniques. The determination is further made without the need to employ linear counting or bias correction operations on low or high cardinalities. An estimated number of unique objects in the obtained multiset is determined as a result of the processing, and subsequently provided for display or further manipulation.