Length-limited data transformation and compression.

摘要

Scientific computation is used for the simulation of increasingly complex phenomena, and generates data sets of ever increasing size, often on the order of terabytes. All of this data creates difficulties. Several problems that have been identified are (1) the inability to effectively handle the massive amounts of data created, (2) the inability to get the data off the computer and into storage fast enough, and (3) the inability of a remote user to easily obtain a rendered image of the data resulting from a simulation run. This dissertation presents several techniques that were developed to address these issues. The first is a prototype bin coder based on variable-to-variable length codes. The codes utilized are created through a process of parse tree leaf merging, rather than the common practice of leaf extension. This coder is very fast and its compression efficiency is comparable to other state-of-the-art coders. The second contribution is the Piecewise-Linear Haar (PLHaar) transform, a reversible n-bit to n-bit wavelet-like transform. PLHaar is simple to implement, ideal for environments where transform coefficients must be kept the same size as the original data, and is the only n-bit to n-bit transform suitable for both lossy and lossless coding.