Re: [RESEND v2] Kbuild: Update config_data.gz only if KCONFIG_CONFIG materially changed

[Elliot Berman]

On 4/24/2021 11:50 PM, Masahiro Yamada wrote:
> On Sat, Apr 24, 2021 at 2:02 AM Elliot Berman <eberman@xxxxxxxxxxxxxx> wrote:
>
>
> Sorry for the delay.

No problem!

> This patch is over-engineering.

I thought so, too.

> I will apply this.
> https://patchwork.kernel.org/project/linux-kbuild/patch/20210425062407.1183801-5-masahiroy@xxxxxxxxxx/
>
> The 'cmp' command is not expensive.
> md5sum is unneeded.

Thanks!

> > If you update the timestamp of KCONFIG_CONFIG without actually changing
> > anything, config_data.gz is re-generated and causes vmlinux to re-link.
> > When Link Time Optimization is enabled, unnecessary re-linking of
> > vmlinux is highly desirable since it adds several minutes to build time.
> >
> > Avoid touching config_data.gz by using a script to compare the existing
> > config_data.gz, KCONFIG_CONFIG, or script itself to update only if any
> > is mismatched.  The script follows gen_kheaders.sh approach for
> > determing in update is needed. The script intentionally avoids
> > re-compressing KCONFIG_CONFIG.
> >
> > The .config can be touched, for instance, by a build script which
> > installs the default defconfig and then applies a defconfig fragment on
> > top.
> >
> > For a simple example on my x86 machine, I modified x86 default defconfig to set
> > CONFIG_IKCONFIG=y and run:
> >    make -j50 defconfig tiny.config vmlinux
> >    make -j50 defconfig tiny.config vmlinux
> > With this patch, vmlinux is not re-built as a result of config_data.gz
> > change.
> >
> > Changes in v2:
> >   - Use md5 checksum to compare .config instead of gzip'ing again
> >
> > Signed-off-by: Elliot Berman <eberman@xxxxxxxxxxxxxx>
> > ---
> >   kernel/.gitignore         |  1 +
> >   kernel/Makefile           |  4 +++-
> >   kernel/gen_config_data.sh | 31 +++++++++++++++++++++++++++++++
> >   3 files changed, 35 insertions(+), 1 deletion(-)
> >   create mode 100755 kernel/gen_config_data.sh
> >
> > diff --git a/kernel/.gitignore b/kernel/.gitignore
> > index 78701ea..a191136 100644
> > --- a/kernel/.gitignore
> > +++ b/kernel/.gitignore
> > @@ -1,4 +1,5 @@
> >   # SPDX-License-Identifier: GPL-2.0-only
> > +config_data.gz.md5
> >   kheaders.md5
> >   timeconst.h
> >   hz.bc
> > diff --git a/kernel/Makefile b/kernel/Makefile
> > index 320f1f3..0784bf3d 100644
> > --- a/kernel/Makefile
> > +++ b/kernel/Makefile
> > @@ -139,8 +139,10 @@ obj-$(CONFIG_SCF_TORTURE_TEST) += scftorture.o
> >   $(obj)/configs.o: $(obj)/config_data.gz
> >
> >   targets += config_data.gz
> > +quiet_cmd_genicfg = CHK     $(obj)/config_data.gz
> > +      cmd_genicfg = $(CONFIG_SHELL) $(srctree)/kernel/gen_config_data.sh $@ $<
> >   $(obj)/config_data.gz: $(KCONFIG_CONFIG) FORCE
> > -       $(call if_changed,gzip)
> > +       $(call cmd,genicfg)
> >
> >   $(obj)/kheaders.o: $(obj)/kheaders_data.tar.xz
> >
> > diff --git a/kernel/gen_config_data.sh b/kernel/gen_config_data.sh
> > new file mode 100755
> > index 00000000..e9ff193
> > --- /dev/null
> > +++ b/kernel/gen_config_data.sh
> > @@ -0,0 +1,31 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# This script generates a compressed version of .config, if its checksum has changed
> > +set -e
> > +
> > +this_file="$(readlink -f "$0")"
> > +outfile=$1
> > +infile=$2
> > +
> > +config_md5="$(md5sum $infile | cut -d ' ' -f1)"
> > +# Any changes to this script will also cause a rebuild of config_data.
> > +this_file_md5="$(md5sum $sfile | cut -d ' ' -f1)"
> > +if [ -f $outfile ]; then outfile_md5="$(md5sum $outfile | cut -d ' ' -f1)"; fi
> > +
> > +if [ -f $outfile.md5 ] &&
> > +       [ "$(head -n 1 $outfile.md5)" = "$config_md5" ] &&
> > +       [ "$(head -n 2 $outfile.md5 | tail -n 1)" = "$this_file_md5" ] &&
> > +       [ "$(tail -n 1 $outfile.md5)" = "$outfile_md5" ]; then
> > +               exit
> > +fi
> > +
> > +if [ "${quiet}" != "silent_" ]; then
> > +       echo "  GEN     $outfile"
> > +fi
> > +
> > +${KGZIP} -c -n -f -9 $infile > $outfile
> > +
> > +echo "$config_md5" > $outfile.md5
> > +echo "$this_file_md5" >> $outfile.md5
> > +echo "$(md5sum $outfile | cut -d ' ' -f1)" >> $outfile.md5
> > --
> > The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> > a Linux Foundation Collaborative Project