Re: [PATCH 1/1] kbuild: deb-pkg: Allow parallel build

[Sedat Dilek]

On Wed, Mar 15, 2023 at 3:24 PM Masahiro Yamada <masahiroy@xxxxxxxxxx> wrote:
>
> On Tue, Mar 14, 2023 at 3:56 AM Bastian Germann <bage@xxxxxxxxxxxxx> wrote:
> >
> > Am 13.03.23 um 18:59 schrieb Masahiro Yamada:
> > > On Tue, Mar 14, 2023 at 2:10 AM Bastian Germann <bage@xxxxxxxxxxxxx> wrote:
> > >>
> > >> Currently, the only way to build the deb-pkg generated package parallely
> > >> is adding -jN to the MAKEFLAGS environment variable. The package ignores
> > >> the usual parallel build option that is described in Debian Policy §4.9.1.
> > >
> > >
> > >
> > > "dpkg-buildpackage -b -j16" worked for me.
> >
> > This ends up in DEB_BUILD_OPTIONS=parallel=16 being set and the call:
> > /usr/bin/make -f ./Makefile ARCH=x86    KERNELRELEASE=6.3.0-rc2 KBUILD_BUILD_VERSION=1  olddefconfig all
>
>
> "dpkb-buildpackage -j<N>" sets not only DEB_BUILD_OPTIONS
> but also MAKEFLAGS.
>
>
> This is clearly explained in "man dpkb-buildpackage".
>
>
>  -j, --jobs[=jobs|auto]
>    ...
>   Will add itself to the MAKEFLAGS environment variable, which should
>   cause all subsequent make invocations to inherit the option, thus
>   forcing the parallel setting on the packaging ...
>
>
>
>
> Your statement sounds like
>
>  'MAKEFLAGS=-j<N> dpkg-buildpackage'
>
> is the only way to build packages in parallel.
>
> Apparently, dpkg-buildpackage provides a much shorter way
> and invokes internal Make in parallel.
>
>
>
>
> >
> > So it is not used to run the actual build, just the top level `make -f debian/rules` invocation.
> > You can set --jobs-force=16, which ends up in MAKEFLAGS but the point of the patch is that one can use
> > the usual way of parallelizing.
>
>
> What is the "usual" way in this context?
>
>
> Do you mean
> 'DEB_BUILD_OPTIONS=parallel=16 dpkg-buildpackage -b'
> is the usual way for parallel building?
>
> If so, I agree. This patch caters to this case.
>
>
> But, I think your code should go to debian/rules
> instead of scripts/package/deb-build-option.
> Kbuild's "make clean" works much faster with the parallel option.
>
>
> Also, the commit description should explain the
> benefit of this patch.
>
>
> >
> > Side note: Without the patch, the build is run with -j1 regardless of being called from the Linux Makefile
> > or dpkg-buildpackage.
>
>
> Try:
>
>  $ make deb-pkg -j16
>
> and
>
>  $ dpkg-buildpackage -b -j16
>
>
> Both run the package building in parallel.
>
> You see a big difference in build speed w/wo the -j option.
> (and the build logs are interleaved, since they
> are emitted by multiple threads)
>
>

I have no strong opinion in having a DEB_BUILD_OPTIONS...

As you stated Masahiro I have seen it mostly in debian/rules.

See the example from [1]:

ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
    NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
    MAKEFLAGS += -j$(NUMJOBS)
endif

Normally, I use in my kernel build-script:

$ make deb-pkg -j${NUMJOBS}

My €0,02.

-Sedat-


[1] https://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-options