ACE: A Resource-Aware Adaptive Compression Environment

Sezgin Sucu and Chandra Krintz

International Conference on Information Technology: Coding and Computing (ITCC03), April 28-30, 2003

Abstract

The growth and evolution of the Internet has created mass demand for always-available, high-performance, and world-wide access by Internet users. These users, as indicated by the popularity of email, search engines, web pages, mobile languages, and systems like Gnutella and others, use the Internet primarily for transfer of data. However, transfer performance is dictated by the underlying resource performance of the Internet and as such, is sensitive to the vast differences in network and end-point technology (wireless v/s ethernet, hand-held v/s desktops) and the high variability in performance (due to contention, protocols used, hardware, etc.). High-performance transfer over the Internet (and hence user satisfaction) is increasingly difficult to obtain.

Compression is commonly used to reduce the number of bytes transferred and to increase the effective bandwidth available. However, there are five primary constraints that must be considered in the selection of a compression technique for Internet transfer: (1) Compression techniques trade space for time; the complexity required to reduce file size impacts the time required for decompression. (2) Compression and decompression time depends on CPU performance and compressed (and uncompressed) transfer time depends on network bandwidth. (3) The characteristics of the data being compressed impact compression performance and it is not possible to learn this effect without first compressing. (4) Commonly, Internet computing applications produce data for transmission dynamically, precluding the use of off-line compression. And (5), compression techniques vary in performance in terms of compression rate and compression ratio. Each of these constraints make it increasingly difficult to identify the ``best'' compression technique in all circumstances.

To address these challenges and to improve Internet transfer performance, we have developed ACE, an Adaptive Compression Environment which automatically and transparently selects between competitive compression techniques (and not compressing at all) based on underlying resource (CPU and network bandwidth) performance. ACE is novel in that it combines existing technologies from Computational Grid Computing and dynamic compilation research to perform adaptive compression. Our results show that ACE can significantly improve transfer time.