Multiplexing H.264 video with AAC audio bit streams, demultiplexing and achieving lip synchronization during playback.

摘要

H.264, MPEG-4 part-10 or AVC [5], is the latest digital video codec standard which has proven to be superior than earlier standards in terms of compression ratio, quality, bit rates and error resilience. However, the standard just defines a video codec and has no mention of any audio compression. In order to have a meaningful delivery of the video to the end user, it is necessary to associate an audio stream along with it. AAC (advanced audio coding) [1] is the latest digital audio codec standard defined in MPEG-2 and later in MPEG-4 with few changes. The audio quality of an AAC stream is observed to be better than both MP3 and AC3, which were widely used as the audio coding standard in various applications, at lower bit rates. Adopting H.264 as video codec and AAC as the audio codec, for transmission of digital multimedia through air (ATSC, DVB) or through the internet (video streaming, IPTV), facilitates the users to take advantage of the leading technologies in both audio and video. However, for these applications, treatment of video and audio as separate streams requires multiplexing the two in order to create a single bit stream for transmission. The objective of the thesis is to propose a method for effectively multiplexing the audio and video coded streams for transmission followed by demultiplexing the streams at the receiving end and achieve lip sync between the audio and video during playback. The proposed method takes advantage of the frame wise arrangement of data in both audio and video codecs. The audio and video frames are used as the first layer of packetization. The frame numbers of the audio and video data blocks are used as the reference for aligning the streams in order to achieve lip sync. The synchronizing information is embedded in the headers of the first layer of packetization. Then second layer of packetization is carried out from the first layer in order to meet the various requirements of transmission channels. Proposed method uses playback time as the criteria for allocating data packets during multiplexing in order to prevent buffer overflow or underflow at the demultiplexer end. More information is embedded into the headers to ensure an effective and fast demultiplexing process, to detect errors and correct them. Advantages and limitations of the proposed method are discussed in detail.