rzip

Rzip is a compression program that functions similarly to bzip2 or gzip. However, it has the unique ability to take advantage of long-distance redundancies in files which can enable it to produce higher compression ratios than other programs. The original concept behind rzip can be found in a PhD thesis, but this current version has undergone considerable development and improvements.

The latest version of rzip is much faster and delivers better compression ratios. The most significant advantage of rzip is that it has an effective history buffer of 900 MB. This means it can match input file pieces much more efficiently than other commonly used compression programs. In contrast, gzip utilizes a history buffer of only 32 KB while bzip2 uses a buffer of just 900 KB.

Another advantage of rzip is that it is generally faster than bzip2. This may appear surprising since rzip uses bzip2's library as a backend for handling short-range compression. However, rzip has usually reduced the data substantially before giving it to bzip2. This means bzip2 has to do less work.

Despite its impressive compression capabilities, rzip may not be suitable for everyone. The two most significant drawbacks are that it cannot be pipelined, so it cannot read from standard input or write to standard output, and it requires a lot of memory. A typical compression run on a large file could need a couple of hundred MB RAM. If your priority is optimal compression rates and you have sufficient RAM, rzip may be your best solution. Otherwise, it's best to use bzip2 or gzip.

Some new features have been added to the latest release. An -L compression level option has been introduced, minor portability fixes have been implemented, and a bug that impeded the decompression of some files has been fixed. Overall, rzip is an excellent compression program that delivers higher compression ratios than competing products in many cases.