Proc. of ADL '96, Forum on Research and Technology Advances in Digital Libraries, IEEE,
Washington D.C., May 1996, pp. 139-148.
D. Andresen, T. Yang, Ö. Egecioglu, O. Ibarra, and T. Smith
Scalability Issues for High Performance Digital Libraries on the World Wide Web
Abstract.
We investigate scalability issues involved in developing
high performance digital library systems. Our observations and
solutions are based on our experience with the Alexandria Digital
Library (ADL) testbed under development at UCSB. The current ADL
system provides on-line browsing and processing of digitized maps and other
geo-spatially mapped data via the World Wide Web (WWW). A primary activity
of the ADL system involves computation and disk I/O for accessing
compressed multi-resolution images with hierarchical data structures,
as well as other duties such as supporting database queries and
on-the-fly HTML page generation. Providing multi-resolution image
browsing services can reduce network traffic but impose some
additional cost at the server. We discuss the necessity of having
a multi-processor DL server to match potentially huge demands in
simultaneous access requests from the Internet. We have developed a
distributed scheduling system for processing DL requests, which actively
monitors the usages of CPU, I/O channels and the interconnection network
to effectively distribute work across processing units to exploit task
and I/O parallelism. We present an experimental study on the performance
of our scheme in addressing the scalability issues arising in ADL
wavelet processing and file retrieval. Our results indicate that the
system delivers good performance on these types of tasks.
omer@cs.ucsb.edu