Extending the prediction error coder of H.264/AVC by a vector quantizer

摘要

The standardized video coding algorithms are based on hybrid coding using blockwise motion compensated prediction and transform coding of the resulting prediction error. For the purpose of transform coding, the recent standard H.264/AVC applies an integer transform. For each block, the Lagrangian costs are analyzed, which are measured by the sum of the squared reconstruction errors and the bit rate weighted by a Lagrange multiplier. It is observed that the costs of blocks with marginally or diagonally correlated samples are frequently higher than the costs theoretically required due to the fact that the transform coder of H.264/AVC is unadjusted for these blocks. In this paper, it is investigated if the coding efficiency can be improved by extending the prediction error coder by a vector quantizer which is optimized for the coding of these blocks. For each block of the prediction error either standardized transform coding or vector quantization is applied whereas the algorithm with lower costs is chosen. For broadcast quality at 34 dB PSNR, the bit rate is reduced by 7-10% compared to H.264/AVC using CAVLC with slightly reduced computational expense in the decoder. Compared to H.264/AVC using CABAC, almost the same coding efficiency is achieved with significantly lower computational expense in the decoder.