Re: Kernel Cross Compiling

From: Herbert Poetzl
Date: Wed Feb 18 2004 - 02:21:55 EST

Hi Judith!

On Tue, Feb 17, 2004 at 01:51:19PM -0800, Judith Lebzelter wrote:
> > tried that one for vs1.20 (2.4.21,2.4.22,2.4.23) was
> > a little inflexible, but I didn't understand the results:
> >
> > PPC-Cross Compile Regress 447 warnings, 21 errors PASS?
>
> The Compile Regress is John Cherry's filter which we run through plm,
> and I include with the plm package. The 'PASS' status is based only
> on the main compiles, due to the fact that some for the compiles of the
> 'defconfig', 'allnoconfig', 'allyesconfig' and 'allmodconfig'. It also
> goes though the directories and compiles the objects individually, but
> this does contribute to fail the test.
>
> There is a -s option to only run 'defconfig' and 'allmodconfig' in the
> newer versions of this script.
>
> > and what I would need is the following:
> >
> > - vanilla kernel + config which compiles for arch X
> > - vanilla kernel + vserver patches + same config
> > which is tested for arch X
> > - differences in the warnings/errors to the default
> > (vanilla) build
> > - ability to check a 'special' config
>
> PLM is as application for tracking base releases and patches to the
> release, on which you can configure 'filters' to run and store the
> results. Each architecture or type of config you want to check could be
> set up as an individual filter. The kconfig files I use are generally
> part of the specific filter which I've set up.

okay, sounds basically useful ...

> The source trees (base + patches) are then also easily available to other
> plm clients. In our case STP.
>
> > > Basically, the farm here at OSDL auto-builds
> > > - each release
> > > - a daily BK snapshot ( from kernel.org )
> > > - each new tree from a few maintainers ( -mm, -ac, -mjb, -osdl )
> >
> > currently there are three linux-vserver branches,
> > stable (2.4), devel (2.4) and experimental (2.6)
> > very soon the experimental branch will become the
> > devel branch, and the devel branch the last stable
> > for (2.4) devel and stable will move on to 2.6 then
> >
> > > We do some cross compiles, using Dan Kegel's fine tools.
> >
> > tried those, but decided not to use them, as they require
> > building the (g)libc which isN#t required for kernel
> > builds at all ...
> >
> > > Several things you/we could do:
> > > - You could of course use our stuff to run your compilie farm, we're
> > > always glad to help. :)
> >
> > okay, how could this be done, and what would be my part
> > especially:
> >
> > - how to add new patches/releases/branches
> > - how to activate a testbuild for a kernel/patch
> > combination
> > - how to use it 'just' for a test, to detect and
> > correct build errors (the main purpose)
> >
>
> To use our PLM:
> * We would need to build the cross-compilers here on our compile farm.
> The cross-compilers would need to be built in an easily reproduced
> manner--to be installed on multiple machines and easily re-built

thought so, that was also my concern in the first place
(see details below)

> * Design/edit filters to suit your purpose with a reasonably short
> run time. We would set them up in PLM.
> * Patch submission is through the PLM web interface:
> https://www.osdl.org/plm-cgi/plm?module=addpatch
> You just upload a text diff file.
> * The results are summarised in the output of the filter which you set
> up and available online.

basically the only missing feature would be building the
differences between a 'vanilla' build run and a vserver
kernel build run, which I think could be done via the
web interface or, if this is interesting for you too,
as addition to the PLM ...

> To Set up your own PLM:
> * You could use our PLM software to set up your own automated
> source/filter tracking system.
> Home: http://developer.osdl.org/dev/plm/
> Download: http://sourceforge.net/projects/plm
> * There is a script to build an rpm in the source. There are some
> admin tools to set-up filters, which you grab from a web server.
> * It runs on a MySQL database.
> * You design/edit filters to suit exactly your purpose.

that would wbe an option, maybe for testing purposes
but as many people pointed out, not as useful ...

> > > - we always need help with the project. Mailing list is
> > > plm-devel@xxxxxxxxxxxxxxxxxxxxxx
> >
> > how could I help?
>
> We are planning quite a few updates to PLM soon, including creating base
> patches from CVS and bk pulls, Storing build information with the patch
> sets, Improving the installation scripts, etc.
>
> If the tool doesn't do all you would like any input would be great too.
> It sounds like maybe a page to display all the recent results of a
> particular filter (ie sparc64_default_compile) would be nice.
>
> >
> > > - we're planning on expanding the number of arch's we cross-compile for,
> > > x86_64 and ppc64 are the next two probable targets. We probably
> > > wouldn't want to add a lot of other archs doing large compiles,
> > > but if you have a simple target, we can add quite a few.
> >
> > currently my cross compile toolchains support the
> > following archs:
> >
> > alpha arm cris hppa hppa64 i386 ia64 m68k mips mips64
> > ppc ppc64 s390 sh sh64 sparc sparc64 v850 x86_64
>
> A very impressive list. :) Do you have automated scripts for setting up
> your toolchains on other systems? Like ours?

sure, made them available some time ago, you can get
the spec files and patches for binutils and gcc at

http://vserver.13thfloor.at/Stuff/Cross/

I'm currently building Dan's toolschains (or better
working on it, as they refuse to compile with my gcc,
which I have to admit, is a little outdated) to
compare my toolchains (gcc) with them, because it was
mentioned that building gcc the way I do it, would
introduce subtle compiler bugs, which I want to verify

of course they can be built without RPMs too, but that
was the simplest way for me atm ...

> > the 'defconfig' only works for a subset of those and
> > some arch have even troubles building without heavy
> > patches ...
> >
> > so planning sounds good, but we are already supporting
> > alpha, hppa64, i386, s390, sparc/64 and x86_64
>
> Nobody had shown much interest in the others, so we set our
> priorities...

okay *showing interest* in at least the vserver
supported platforms alpha, hppa64, s390, sparc/64
and x86_64 (ia32/64 and ppc/64 are already supported)

TIA,
Herbert

> Thanks;
> Judith Lebzelter
> OSDL
>
> >
> > > ( we noticed you are doing a defconfig. We've done separate config
> > > files for those builds, to reduce time )
> >
> > sounds like a reasonable approach ...
> >
> > > hope to hear from you
> > > cliffw
> >
> > TIA,
> > Herbert
> >
> > > > TIA,
> > > > Herbert
> > > >
> > > > > Tim
> > > > >
> > > > > On Fri, 2004-02-13 at 12:57, Herbert Poetzl wrote:
> > > > > > Hi Folks!
> > > > > >
> > > > > > I'm currently investigating the requirements/doability
> > > > > > of a kernel cross compiling test bed/setup, able to do
> > > > > > automated kernel builds for different architecture,
> > > > > > just to see if it compiles and later to verify if a
> > > > > > given patch breaks that compile on any of the tested
> > > > > > archs ...
> > > > > >
> > > > > > here a short status, and some issues I ran into so far,
> > > > > > some of them with solutions, others without, and some
> > > > > > interesting? observations ...
> > > > > >
> > > > > > I would be happy if somebody who has done similar, or
> > > > > > knows how to do it properly ;) could comment on that,
> > > > > > and/or point out possible improvements ...
> > > > > >
> > > > > > TIA,
> > > > > > Herbert
> > > > > >
> > > > > >
> > > > > > 1) CROSS COMPILER / TOOLCHAIN
> > > > > >
> > > > > > after reading and testing several cross build and
> > > > > > toolchain building howtos, I decided to do it a little
> > > > > > different, because I do not need glibc to compile the
> > > > > > kernel ...
> > > > > >
> > > > > > the result are two .spec files[1], or the commands
> > > > > > used to build an appropriate toolchain ...
> > > > > >
> > > > > > for the binutils the required commands are:
> > > > > >
> > > > > > configure \
> > > > > > --disable-nls \
> > > > > > --prefix=/usr \
> > > > > > --mandir=/usr/share/man \
> > > > > > --infodir=/usr/share/info \
> > > > > > --target=${CROSS_ARCH}-linux
> > > > > > make
> > > > > >
> > > > > > and for the gcc (after the binutils have been
> > > > > > installed on the host):
> > > > > >
> > > > > > configure \
> > > > > > --enable-languages=c \
> > > > > > --disable-nls \
> > > > > > --disable-threads \
> > > > > > --disable-shared \
> > > > > > --disable-checking \
> > > > > > --prefix=/usr \
> > > > > > --mandir=/usr/share/man \
> > > > > > --infodir=/usr/share/info \
> > > > > > --target=${CROSS_ARCH}-linux
> > > > > > make TARGET_LIBGCC2_CFLAGS='-Dinhibit_libc \
> > > > > > -D__gthr_posix_h'
> > > > > >
> > > > > > where ${CROSS_ARCH} is the target architecture you want
> > > > > > to compile the toochain for, in my case, this where one
> > > > > > of the following:
> > > > > >
> > > > > > alpha, hppa, hppa64, i386, ia64, m68k, mips,
> > > > > > mips64, ppc, ppc64, s390, sparc, sparc64, x86_64
> > > > > >
> > > > > > PROBLEMS HERE:
> > > > > >
> > > > > > I decided to use binutils 2.14.90.0.8, and gcc 3.3.2,
> > > > > > but soon discovered that gcc-3.3.2 will not be able
> > > > > > to build a cross compiler for some archs like the
> > > > > > alpha, ia64, powerpc and even i386 ;) without some
> > > > > > modifications[2] but with some help, I got all headers
> > > > > > fixed, except for the ia64, which still doesn't work
> > > > > >
> > > > > >
> > > > > > 2) KERNEL CROSS COMPILING
> > > > > >
> > > > > > equipped with the cross compiling toolchains for all
> > > > > > but one of the architectures mentioned above, I wrote
> > > > > > a little script, which basically does nothing else
> > > > > > but compiling a given kernel for all possible archs.
> > > > > >
> > > > > > basically this can be accomplished by doing:
> > > > > >
> > > > > > make ARCH=<arch> CROSS_COMPILE=<arch>-linux-
> > > > > >
> > > > > >
> > > > > > the first result was harrowing:
> > > > > >
> > > > > > 2.4.25-pre 2.6.2-rc
> > > > > > ----------------------------------------------------
> > > > > > [ARCH alpha/alpha] succeeded. succeeded.
> > > > > > [ARCH hppa/parisc] failed. failed.
> > > > > > [ARCH hppa64/parisc] failed. failed.
> > > > > > [ARCH i386/i386] succeeded. succeeded.
> > > > > > [ARCH m68k/m68k] failed. failed.
> > > > > > [ARCH mips/mips] failed. failed.
> > > > > > [ARCH mips64/mips] failed. failed.
> > > > > > [ARCH ppc/ppc] succeeded. succeeded.
> > > > > > [ARCH ppc64/ppc64] failed. failed.
> > > > > > [ARCH s390/s390] failed. failed.
> > > > > > [ARCH sparc/sparc] failed. succeeded.
> > > > > > [ARCH sparc64/sparc] failed. failed.
> > > > > > [ARCH x86_64/x86_64] failed. succeeded.
> > > > > >
> > > > > > so only alpha, i386 and ppc did work on the first run.
> > > > > >
> > > > > > what I discovered was, that there IS a big difference
> > > > > > between an empty .config file and a non exististing
> > > > > > one, where latter allowed the make oldconfig to work
> > > > > > similar to the make defaultconfig available on 2.6,
> > > > > > and added some archs (see [3] for details)
> > > > > >
> > > > > > PROBLEMS & SOLUTIONS HERE:
> > > > > >
> > > > > > ppc64:
> > > > > > CROSS32_COMPILE=ppc-linux-
> > > > > > is needed to make this work as expected.
> > > > > >
> > > > > > hppa/hppa64:
> > > > > > seems not to compile without using a very big
> > > > > > patch, which changes a lot inside the kernel
> > > > > >
> > > > > > mips/mips64:
> > > > > > seem to use the 'obsoleted' -mcpu= option
> > > > > > which results in a cc1: error: invalid option
> > > > > > `cpu=<cpu-here>'
> > > > > >
> > > > > > m68k:
> > > > > > fails with a hundred errors in the includes
> > > > > >
> > > > > >
> > > > > > 3) CONCLUSIONS
> > > > > >
> > > > > > it seems that recent kernels (2.4 and 2.6) do not support
> > > > > > most of the architectures they contain without heavy
> > > > > > patching (haven't tested for arm, sh3/4, ...)
> > > > > >
> > > > > > building cross compiler toolchains isn't that often done
> > > > > > otherwise it would not require such modifications, and
> > > > > > the documentation would be up to date ...
> > > > > >
> > > > > > it seems that with some minor patches and kernel tweaks
> > > > > > an automated build is in reach, although some archs seem
> > > > > > to break from one release to the other ...
> > > > > >
> > > > > > the non mainline branches, if they exist are some kernel
> > > > > > versions behind the current mainstream kernel, which
> > > > > > might not mean anything ...
> > > > > >
> > > > > >
> > > > > > 4) LINKS & REFERENCES
> > > > > >
> > > > > > [1] http://vserver.13thfloor.at/Stuff/Cross/binutils-cross.spec
> > > > > > http://vserver.13thfloor.at/Stuff/Cross/gcc-cross.spec
> > > > > >
> > > > > > [2] http://vserver.13thfloor.at/Stuff/Cross/
> > > > > > gcc-3.3.2-cross-alpha-fix.diff.bz2
> > > > > > gcc-3.3.2-cross-i386-fix.diff.bz2
> > > > > > gcc-3.3.2-cross-ia64-fix.diff.bz2
> > > > > > gcc-3.3.2-cross-powerpc-fix.diff.bz2
> > > > > >
> > > > > > [3] http://vserver.13thfloor.at/Stuff/Cross/compile.info
> > > > > >
> > > > > > ia64: http://www.gelato.unsw.edu.au/kerncomp/
> > > > > > mips: http://www.linux-mips.org/kernel.html
> > > > > > hppa: http://www.parisc-linux.org/kernel/index.html
> > > > > > ppc64: http://linuxppc64.org/
> > > > > >
> > > > > >
> > > > > > -
> > > > > > To unsubscribe from this list: send the line "unsubscribe linux-gcc" in
> > > > > > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > > > --
> > > > > Timothy D. Witham - Lab Director - wookie@xxxxxxxx
> > > > > Open Source Development Lab Inc - A non-profit corporation
> > > > > 12725 SW Millikan Way - Suite 400 - Beaverton OR, 97005
> > > > > (503)-626-2455 x11 (office) (503)-702-2871 (cell)
> > > > > (503)-626-2436 (fax)
> > > > >
> > > > > -
> > > > > To unsubscribe from this list: send the line "unsubscribe linux-gcc" in
> > > > > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > > -
> > > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > > Please read the FAQ at http://www.tux.org/lkml/
> > > >
> > >
> > >
> > > --
> > > The church is near, but the road is icy.
> > > The bar is far, but i will walk carefully. - Russian proverb
> >
> >
> >
> >
>
>
>
>
>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/