FLASH OPTIMIZED COLUMNAR DATA LAYOUT AND DATA ACCESS ALGORITHMS FOR BIG DATA QUERY ENGINES

摘要

A technique relates to flash-optimized data layout of a dataset for queries. Selection columns are stored in flash memory according to a selection optimized layout, where the selection optimized layout is configured to optimize predicate matching and data skipping. The selection optimized layout, for each selection column, is formed by storing a selection column dictionary filled with unique data values in a given selection column, where the unique data values are stored in sorted order in the selection column dictionary. Row position designations are stored corresponding to each row position that the unique data values are present within the given selection column, without duplicating storage of any of the unique data values that occur more than once in the given selection column.