Adaptive On-The-Fly Compression

IEEE Transactions on Parallel and Distributed Systems (TPDS)
Vol. 17, Number 1, January, 2006

Chandra Krintz and Sezgin Sucu



We present a system called the Adaptive Compression Environment (ACE) that automatically and transparently applies compression (on-the-fly) to a communication stream to improve network transfer performance. ACE uses a series of estimation techniques to make short-term forecasts of compressed and uncompressed transfer time at the block level. ACE considers underlying networking technology, available resource performance, and data characteristics as part of its estimations to determine which compression algorithm to apply (if any). Our empirical evaluation shows that, on average, ACE improves transfer performance given changing network types and performance characteristics by 8 to 93% over using any of the popular compression techniques that we studied (Bzip, Zlib, LZO, and no compression) alone.