Efficient CRC generation utilizing parallel table lookup operations

摘要

An improved CRC generation mechanism for generating a CRC value of relevant data in a digital data stream is disclosed wherein relevant data in the data stream is identified and partitioned into a plurality of intervals. A CRC value is determined for each interval by partitioning the interval into a plurality of chunks, loading from persistent storage a table of CRC values for a range of chunks, determining a CRC value for each of the chunks with parallel table lookup operations on the table, and combining the CRC values for the chunks. The CRC values for each of the intervals is combined to generate the CRC for the relevant data. The parallel table look operation is preferably a vector permute instruction that is executed by a SIMD-style vector unit.