Re: [PATCH] kbuild: disable pahole multithreading for reproducible builds

[Alan Maguire]

On 22/03/2024 08:31, Thomas Weißschuh wrote:
> A BTF type_id is a numeric identifier allocated by pahole through
> libbpfd. Ids are incremented for each allocation.
> Running pahole multithreaded makes the sequence of allocations
> non-deterministic which also makes the type_id itself non-deterministic.
> As the type_id end up in the binary this breaks reproducibility.
> 
> Therefore, if the kernel is built reproducibly as indicated by
> KBUILD_BUILD_TIMESTAMP, disable threading in pahole.
> 
> Fixes: b4f72786429c ("scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading.")
> Fixes: 72d091846de9 ("kbuild: avoid too many execution of scripts/pahole-flags.sh")
> Signed-off-by: Thomas Weißschuh <linux@xxxxxxxxxxxxxx>

Yep, we've talked about sorting final BTF in various ways but until that
is implemented, non-parallel execution of pahole is the best way to get
reliabily-reproducing BTF ids. We should probably update the pahole
manpage to warn about parallel non-deterministic id allocation in the
meantime.

Reviewed-by: Alan Maguire <alan.maguire@xxxxxxxxxx>

> ---
>  scripts/Makefile.btf | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
> index 82377e470aed..2356929e62e4 100644
> --- a/scripts/Makefile.btf
> +++ b/scripts/Makefile.btf
> @@ -10,7 +10,10 @@ endif
>  
>  pahole-flags-$(call test-ge, $(pahole-ver), 121)	+= --btf_gen_floats
>  
> +# threaded execution randomizes BTF type_id allocation, breaking reproducibility
> +ifeq ($(KBUILD_BUILD_TIMESTAMP),)
>  pahole-flags-$(call test-ge, $(pahole-ver), 122)	+= -j
> +endif
>  
>  pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE)		+= --lang_exclude=rust
>  
> 
> ---
> base-commit: 8e938e39866920ddc266898e6ae1fffc5c8f51aa
> change-id: 20240322-pahole-reprodicible-02e904e45686
> 
> Best regards,