A Subzone-Based Client-Server Technique for I/O Efficient Analysis and Visualization of Large Remote Datasets

摘要

The size and number of datasets analyzed by post-processing and visualization tools are growing with Moore's law. Conversely, the disk-read data transfer rate is only doubling every 36 months and is destined to be the bottleneck for traditional post-processing architectures. To eliminate this bottleneck during loading post-processing visualization and analysis, a subzone load-on-demand (SZL) visualization architecture has been developed which loads only the data needed to create the desired plot. The same I/O bottleneck also affects the transfer of datasets from remote to local storage systems. In this paper, the SZL technique is used to create a client-server architecture where only the volume subzones needed to create the plot are transferred from the remote computer. The new technique differs from traditional client-server architectures where surface-data, such as slices or isosurfaces, are extracted by the remote server and transferred to the local client. The subzone-based client-server method transfers more data initially but can re-use the transferred volume subzones for other data extractions during further exploration. The new technique was tested with slices and isosurfaces. The initial data transfer from the server was found to be 3-17 times more than equivalent plots that transfer only surfaces, but one to two orders of magnitude less than the full volume data size. Server memory usage was substantially reduced compared with traditional client-server techniques.