Processing device utilizing polynomial-based signature subspace for efficient generation of deduplication estimate

摘要

An apparatus in one embodiment comprises at least one processing device comprising a processor coupled to a memory. The processing device is configured to identify a dataset to be scanned to generate a deduplication estimate for that dataset, to designate a subset inclusion characteristic to be utilized in the scan, and for each of a plurality of pages of the dataset, to scan the page, where scanning the page includes computing a polynomial-based signature for the page, determining whether or not the polynomial-based signature satisfies the designated subset inclusion characteristic, and responsive to the polynomial-based signature satisfying the designated subset inclusion characteristic, computing a content-based signature for the page and updating a corresponding entry of a deduplication estimate table for the dataset based at least in part on the content-based signature. The processing device generates the deduplication estimate for the dataset based at least in part on contents of the deduplication estimate table.