System, method, and computer-readable medium for partial redistribution, partial duplication of rows of parallel join operation on skewed data

摘要

A system, method, and computer-readable medium that facilitate management of data skew during a parallel join operation are provided. Portions of tables involved in the join operation are distributed among a plurality of processing modules, and each of the processing modules is provided with a list of skewed values of a join column of a larger table involved in the join operation. Each of the processing modules scans the rows of the tables distributed to the processing modules and compares values of the join columns of both tables with the list of skewed values. Rows of the larger table having non-skewed values in the join column are redistributed, and rows of the larger table having skewed values in the join column are maintained locally at the processing modules. Rows of the smaller table that have non-skewed values in the join column are redistributed, and rows of the smaller table that have skewed values in the join column are duplicated among the processing modules.