Re: [PATCH] init: bzip2 or lzma -compressed kernels and initrds

From: Rob Landley
Date: Sun Sep 07 2008 - 02:18:22 EST


On Saturday 06 September 2008 17:59:25 Alain Knaff wrote:
> Leon Woestenberg wrote:
> > Hello,
> >
> > a small remark on the non-code parts:
> >
> > On Sat, Sep 6, 2008 at 11:19 PM, Alain Knaff <alain@xxxxxxxx> wrote:
> >> compressing the kernel with bzip2 or lzma rather than gzip. Both
> >> compressors give smaller sizes than gzip. Moreover, lzma's
> >> decompresses faster than gzip.
> >
> > versus
> >
> >> +config KERNEL_GZIP
> >> + bool "Gzip"
> >> + help
> >> + The old and tried gzip compression. Its compression ratio is
> >> + the poorest among the 3 choices; however its speed (both
> >> + compression and decompression) is the fastest.
> >> +
> >
> > This seems contradictionary information.
>
> Oops, sorry for that. Actually the Kconfig text is correct. On
> decompression, Lzma is faster than bzip2 but slower than gzip:
>
> Compressor Compression Decompression Size
> gzip -9 1,01s 0,11s 833069
> lzma -9 3,43s 0,24s 705125
> bzip2 -9 2,88s 0,38s 777706
>
> On compression, lzma is actually slowest of the 3, but that should be of
> little concern, as this happens only once, whereas decompression happens
> many times (on each boot).
>
> So, overall lzma looks like the best (acceptable decompression speed,
> best decompression ratio). I only included Bzip2 because it's much
> better known than lzma.

I vaguely recall that lzma requires more memory to decompress than bzip2 does,
although I don't remember the details. I know that bzip2 takes around 4 megs
(although you need space for the decompressed kernel on _top_ of that, so you
should be able to do it in about 8 megs total). gunzip uses a 64k sliding
window plus dictionary and the whole mess should fit in about 1/4 of a meg.

Actually, from what I've seen the main reason lzma doesn't get used for
tarballs a lot is that whoever originally created it didn't include a
fingerprint. You can go "file tar.gz" or "file tar.bz2" and it can figure
out by looking at the contents of the file what it _is_, but last I checked
there's no obvious way to tell an lzma file from the output of /dev/urandom.
This causes all sorts of small but annoying problems, and discourages its use
a bit...

(Also, the decompressor's not so bad but the compressor was _painfully_ slow,
last I checked. It's been a couple years since I was really paying
attention, though...)

> Unfortunately, I didn't have any such machine available for testing, so
> I just for Intel 32/64.

Eh, anybody can mess with arm, powerpc, and mips.

1) Install qemu 0.9.1.

2) Go to http://landley.net/code/firmware/downloads

In the binaries/system-image directory are tarballs of prebuilt systems with
zImage files of kernels qemu can boot, and ext2 image files of uClibc/busybox
root filesystems. There are also ./run-emulator.sh scripts that will boot
the above under qemu 0.9.1 (giving you a shell prompt).

If you'd like to replace the zImage files with ones you build yourself, you'll
need to either grab the source tarballs at the top level and run the whole
build yourself (try "./build.sh armv4l" for example; you can run it with no
arguments to see available targets) or grab the appropriate tarball out of
binaries/cross-compiler/host-x86_64, extract it, add its "bin" subdirectory
to your $PATH, and then build the kernel via something like:

make CROSS_COMPILE=armv4l- ARCH=arm

With the appropriate .config of course (look at the defconfig files under
arch/arm/config or grab mine out of the source). I'm using arm as an example
because it's pretty much the most popular embedded processor and has 90%
share in the cell phone market. FWL also supports powerpc, mips, and so on.
(Sparc's boots but won't give you a shell prompt, it's a uClibc 0.9.29 bug.
sh4 isn't properly supported by qemu yet, hopefully next release.)

Feel free to email me if you've got any questions, this is a hobby of mine. :)

Rob
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/