Re: Linux Kernel Dump Summit 2005

From: OBATA Noboru
Date: Wed Oct 12 2005 - 03:29:50 EST

Hi, Pavel,

On Mon, 10 Oct 2005, Pavel Machek wrote:
> > Compressing a 32GB dump file took about 40 minutes on Pentium 4
> > Xeon 3.0GHz, which is not good enough because the dump without
> > compression took only 5 minutes; eight times slower.
> probably want to look at project. They have
> special compressor aimed at compressing exactly this kind of data,
> fast enough to be improvement.

Thank you for pointing me to the interesting project.

I looked at their patch to find that the special compressor uses
the LZF compression algorithm. (Correct me if I'm wrong.)

So I made a quick comparison between cp, lzf(*) and gzip as
follows. The INFILE is a file that showed the worst compression
ratio last time.

(*) A simple compress tool based on LZF algorithm included in
liblzf, available from

$ cp INFILE /dev/null # this puts whole INFILE in a cache

$ /usr/bin/time cp INFILE /mnt/OUTFILE-cp
0.23user 14.16system 0:35.94elapsed 40%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (115major+15minor)pagefaults 0swaps

$ /usr/bin/time ./lzf -c < INFILE > /mnt/OUTFILE-lzf
35.04user 13.10system 0:54.30elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (84major+74minor)pagefaults 0swaps

$ /usr/bin/time gzip -1c < INFILE > /mnt/OUTFILE-gzip-1
186.84user 11.73system 3:20.36elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (90major+93minor)pagefaults 0swaps

The results can be summarized as follows. The lzf tool is
slower than cp (normal file copy) only by a factor of 1.5,
achieving the compress ratio close to gzip -1.

CMD | NET TIME (in seconds) | OUTPUT SIZE (in bytes)
cp | 35.94 (usr 0.23, sys 14.16) | 2,121,438,352 (100.0%)
lzf | 54.30 (usr 35.04, sys 13.10) | 1,959,473,330 ( 92.3%)
gzip -1 | 200.36 (usr 186.84, sys 11.73) | 1,938,686,487 ( 91.3%)

Although it is too early to say lzf's compress ratio is good
enough, its compression speed is impressive indeed. And the
result also suggests that it is too early to give up the idea of
full dump with compression.

Pavel, thank you again for your advice and I'll welcome further

OBATA Noboru (

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at