Re: [PATCH 3/5] kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}

[Masahiro Yamada]

On Tue, Jul 25, 2023 at 6:24 PM Eugeniu Rosca <erosca@xxxxxxxxxxxxxx> wrote:
>
> Hello Yamada-san,
>
> Appreciate your willingness to support. Some findings below.
>
> On Sun, Jul 23, 2023 at 01:08:46AM +0900, Masahiro Yamada wrote:
> > On Thu, Jul 20, 2023 at 4:09 AM Eugeniu Rosca <erosca@xxxxxxxxxxxxxx> wrote:
> > > On Fri, Jun 23, 2023 at 04:45:44PM +0200, Eugeniu Rosca wrote:
>
> [..]
>
> > > > I will continue to increase my understanding behind what's happening.
> > > > In case there are already any suggestions, would appreciate those.
> > >
> > > JFYI, we've got confirmation from Qualcomm Customer Support interface
> > > that reverting [1] heals the issue on QC end as well. However, it looks
> > > like none of us has clear understanding how to properly
> > > troubleshoot/trace/compare the behavior before and after the commit.
> > >
> > > I would happily follow any suggestions.
> > >
> > > > [1] https://android.googlesource.com/kernel/common/+/bc6d3d83539512
> > > >     ("UPSTREAM: kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}")
> > > >
> > > > [2] https://lore.kernel.org/linux-kbuild/20230616194505.GA27753@lxhi-065/
>
> [..]
>
> > Please backport 64d8aaa4ef388b22372de4dc9ce3b9b3e5f45b6c
> > and see if the problem goes away.
>
> Unfortunately, the problem remains after backporting the above commit.
>
> After some more bisecting and some more trial-and-error, I finally came
> up with a reproduction scenario against vanilla. It also shows that
> after reverting 7ce7e984ab2b21 ("kbuild: rename
> cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}"), the problem goes away.
>
> It takes <30 seconds to reproduce the issue on my machine (on 2nd run).
>
> In order to make the test self-sufficient, it also clones the Linux
> sources (only during 1st run, with --depth 1, for minimal footprint),
> hence ~1.8 GB free space is required in /tmp .
>
> The repro.sh script:
>  => https://gist.github.com/erosca/1372fdc24126dc98031444613450c494
>
> Output against vanilla on 1st run (always OK, matches real-life case):
>  => https://gist.github.com/erosca/0f5b8e0a00a256d80f0c8a6364d81568
>
> Output against vanilla on 2nd/Nth run (NOK: Argument list too long):
>  => https://gist.github.com/erosca/e5c2c6479cc32244cc38d308deea4cf5
>
> Output against vanilla + revert_of_7ce7e984ab2b2 on Nth run (always OK):
>  => https://gist.github.com/erosca/57e114f92ea20132e19fc7f5a46e7c65
>
> Would it be possible to get your thoughts on the above?
>
> --
> Best regards,
> Eugeniu Rosca





Indeed, reverting 7ce7e984ab2b218d6e92d5165629022fe2daf9ee
makes qcom's external module build successfully
(but rebuilding is super slow).

Interestingly, revert 7ce7e984ab2b218d6e92d5165629022fe2daf9ee
then apply the attached patch, then
'Argument list too long' will come back.

So, this is unrelated to the actual build commands.




I suspect bare 'export', which expands all variables
while apparently most of them are not meant exported.



Insert the following in your reproducer, then it will work.


# qcom's audio-kernel sprinkles 'export' everywhere.
# Remove bare use of 'export'
find "$ABS_KMOD" -name Kbuild | xargs sed -i '/export$/d'




--
Best Regards
Masahiro Yamada
From a1d641238851dbde00a37b7b203b110e786edd1d Mon Sep 17 00:00:00 2001
From: Masahiro Yamada <masahiroy@xxxxxxxxxx>
Date: Wed, 2 Aug 2023 18:11:32 +0900
Subject: [PATCH] Add dummy commands to make qcom's external module fail

Revert 7ce7e984ab2b218d6e92d5165629022fe2daf9ee and
apply this instead.

I see

/bin/sh: Argument list too long

Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx>
---
 scripts/Makefile.lib | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 2270ed819a29..c481018d1b5f 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -559,3 +559,36 @@ define filechk_offsets
 	 echo ""; \
 	 echo "#endif"
 endef
+
+
+# These dummy commands are not used anywhere, but
+# make qcom's external module fail to build.
+# qcom's Makefile use bare 'export'.
+# Having more and more shell commands may flood variable expansions.
+
+quiet_cmd_dummy1 = FOO    $@
+      cmd_dummy1 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy2 = FOO    $@
+      cmd_dummy2 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy3 = FOO    $@
+      cmd_dummy3 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy4 = FOO    $@
+      cmd_dummy4 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy5 = FOO    $@
+      cmd_dummy5 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy6 = FOO    $@
+      cmd_dummy6 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy7 = FOO    $@
+      cmd_dummy7 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy8 = FOO    $@
+      cmd_dummy8 = cat $(real-prereqs) > $@
+
+quiet_cmd_dummy9 = FOO    $@
+      cmd_dummy9 = cat $(real-prereqs) > $@
-- 
2.39.2