Re: [PATCH] kbuild: prevent building single targets twice

[Mathias Krause]

On 23.09.26 14:39, Mathias Krause wrote:
> When building individual targets via 'make path/to/target.o', target.o
> may be build multiple times concurrently, leading to build errors like
> below:
> 
> $ make O=~/O/vanilla-7.1/ -j$(nproc) arch/x86/pci/intel_mid.o
> make[1]: Entering directory '/scratch/minipli/obj/vanilla-7.1'
>   DESCEND objtool
>   DESCEND bpf/resolve_btfids
>   CC      arch/x86/pci/intel_mid.o
>   CC      arch/x86/pci/intel_mid.o
> fixdep: error opening file: arch/x86/pci/.intel_mid.o.d: No such file or directory
> make[4]: *** [/scratch/minipli/src/linux/scripts/Makefile.build:290: arch/x86/pci/intel_mid.o] Error 2
> make[4]: *** Deleting file 'arch/x86/pci/intel_mid.o'
> make[3]: *** [/scratch/minipli/src/linux/scripts/Makefile.build:551: arch/x86] Error 2
> make[3]: *** Waiting for unfinished jobs....
> make[2]: *** [/scratch/minipli/src/linux/Makefile:2229: .] Error 2
> make[1]: *** [/scratch/minipli/src/linux/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/scratch/minipli/obj/vanilla-7.1'
> make: *** [Makefile:248: __sub-make] Error 2
> 
> Please note that arch/x86/pci/intel_mid.o gets output twice and, in
> fact, tried to get built twice, causing the two parallel builds to
> interfere with each other.
> 
> The reason for the double build attempt is that the target can be
> reached twice, via arch/x86/'s Makefile and via arch/x86/pci/'s which
> both end up in '$(single-subdirs)' as both fit the prefix match.
> 
> Only the latter should be used as it contains a cflags override for
> CONFIG_PCI_DEBUG=y which would be skipped when using arch/x86/Makefile
> to build arch/x86/pci/intel_mid.o.
> 
> Fix the multi-match issue by using only the deepest match instead of all
> possible ones, which requires two changes:
> 1. Limiting 'single-subdirs' to a single match per target and
> 2. filtering out deeper targets in the recursive make invocation.
> 
> Also de-duplicate the list of target subdirectories to simplify the
> use of '$(single-subdirs)' in filter expressions for multiple targets in
> the same directory.
> 
> Fixes: cc306abd19e8 ("kbuild: fix and refactor single target build")
> Assisted-by: LLM
> Signed-off-by: Mathias Krause <minipli@xxxxxxxxxxxxxx>
> ---
> I ran into this quite a few times in the past and only now gave the LLM
> a try to dig into it. The bug only affects certain targets and only when
> mentioned on the command line, so regular 'make -j$(nproc)' isn't
> affected.
> 
>  scripts/Makefile.build | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 0f4fdc119e58..d5f58513d02a 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -534,7 +534,10 @@ endif
>  # Single targets
>  # ---------------------------------------------------------------------------
>  
> -single-subdirs := $(foreach d, $(subdir-ym), $(if $(filter $d/%, $(MAKECMDGOALS)), $d))
> +# A single target may match nested entries in subdir-ym.  Descend through only
> +# the deepest match so that two recursive makes do not build the same target.
> +single-subdir = $(lastword $(sort $(foreach d, $(subdir-ym), $(if $(filter $d/%, $(1)), $d))))
> +single-subdirs := $(sort $(foreach g, $(MAKECMDGOALS), $(call single-subdir, $g)))
>  single-subdir-goals := $(filter $(addsuffix /%, $(single-subdirs)), $(MAKECMDGOALS))
>  
>  $(single-subdir-goals): $(single-subdirs)
> @@ -548,7 +551,8 @@ $(subdir-ym):
>  	$(Q)$(MAKE) $(build)=$@ \
>  	need-builtin=$(if $(filter $@/built-in.a, $(subdir-builtin)),1) \
>  	need-modorder=$(if $(filter $@/modules.order, $(subdir-modorder)),1) \
> -	$(filter $@/%, $(single-subdir-goals))
> +	$(filter-out $(addsuffix /%, $(filter $@/%, $(single-subdirs))), \
> +		$(filter $@/%, $(single-subdir-goals)))
>  
>  # Add FORCE to the prerequisites of a target to force it to be always rebuilt.
>  # ---------------------------------------------------------------------------

Ping! I know, it's the merge window but this patch may have fallen
through the cracks as there was no response so far. Just trying to take
care it doesn't get lost.

Thanks,
Mathias