Scalability issues for high performance digital libraries on theWorld Wide Web

摘要

We investigate scalability issues involved in developing highperformance digital library systems. Our observations and solutions arebased on our experience with the Alexandria Digital Library (ADL)testbed under development at UCSB. The current ADL system providesonline browsing and processing of digitized maps and other geospatiallymapped data via the World Wide Web (WWW). A primary activity of the ADLsystem involves computation and disk I/O for accessing compressed multiresolution images with hierarchical data structures, as well as otherduties such as supporting database queries and on the fly HTML pagegeneration. Providing multi resolution image browsing services canreduce network traffic but impose some additional cost at the server. Wediscuss the necessity of having a multiprocessor DL server to matchpotentially huge demands in simultaneous access requests from theInternet. We have developed a distributed scheduling system forprocessing DL requests, which actively monitors the usages of CPU, I/Ochannels and the interconnection network to effectively distribute workacross processing units to exploit task and I/O parallelism. We presentan experimental study on the performance of our scheme in addressing thescalability issues arising in ADL wavelet processing and file retrieval.Our results indicate that the system delivers good performance on thesetypes of tasks