Re: [PATCH 02/15] scripts/sbom: integrate script in make process

[Luis Augenstein]

Hi Nathan,

thanks a lot for your recommendations.

> Does sbom-roots.txt need to be cleaned up as well?

This file is only required to pass the roots into the python script.
We could also use a tmp file. Then we don't need to worry about clean
up. Together with your other suggested changes something like this
should work:

# Script to generate .spdx.json SBOM documents describing the build
#
---------------------------------------------------------------------------

ifdef building_out_of_srctree
sbom_targets := sbom-source.spdx.json
endif
sbom_targets += sbom-build.spdx.json sbom-output.spdx.json
quiet_cmd_sbom = GEN     $(notdir $(sbom_targets))
      cmd_sbom = roots_file=$$(mktemp); \
                 printf "%s\n" "$(KBUILD_IMAGE)" >"$$roots_file"; \
                 $(if $(CONFIG_MODULES),sed 's/\.o$$/.ko/'
$(objtree)/modules.order >> "$$roots_file";) \
                 $(PYTHON3) $(srctree)/scripts/sbom/sbom.py \
                     --src-tree $(abspath $(srctree)) \
                     --obj-tree $(abspath $(objtree)) \
                     --roots-file "$$roots_file" \
                     --output-directory $(abspath $(objtree)) \
                     --generate-spdx \
                     --package-license "GPL-2.0 WITH Linux-syscall-note" \
                     --package-version "$(KERNELVERSION)" \
                     --write-output-on-error;
                 rm -f "$$roots_file"
PHONY += sbom
sbom: $(notdir $(KBUILD_IMAGE)) include/generated/autoconf.h $(if
$(CONFIG_MODULES),modules modules.order)
	$(call cmd,sbom)

Note, I will also add the --write-output-on-error flag by default such
that the .spdx.json documents are generated as much as possible even if
some build commands are unknown to the parser.

> FWIW, I get errors like
>
>   $ make -kj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux- O=build
mrproper defconfig sbom
>   ...
>     GEN     sbom-source.spdx.json sbom-build.spdx.json
sbom-output.spdx.json
>   [ERROR] File "/src/scripts/sbom/sbom/cmd_graph/savedcmd_parser.py",
line 630, in log_error_or_warning
>   Skipped parsing command ccache aarch64-linux-gcc ... -o init/main.o
/src/init/main.c because no matching parser was found
>   [ERROR] File "/src/scripts/sbom/sbom/cmd_graph/savedcmd_parser.py",
line 630, in log_error_or_warning
>   Skipped parsing command ccache aarch64-linux-gcc ... -o
arch/arm64/kernel/asm-offsets.s /src/arch/arm64/kernel/asm-offsets.c
because no matching parser was found
>   [ERROR] File "/src/scripts/sbom/sbom/cmd_graph/savedcmd_parser.py",
line 630, in log_error_or_warning
>   Skipped parsing command ccache aarch64-linux-gcc ... -o
kernel/bounds.s /src/kernel/bounds.c because no matching parser was found
>   ... (Found 10435 more instances of this error)
>
> when testing the whole series without any modifications, am I doing
> something wrong?

I was not aware of ccache. If you rebuild without using ccache the gcc
commands should be parsed correctly.

The parser expects gcc commands to be of the form
	"^([^\s]+-)?(gcc|clang)\b"
When using tools like ccache this breaks. I will update the parser to
look for
	"^(ccache\s+)?([^\s]+-)?(gcc|clang)\b"
instead.

Feedback like this is very helpful—thanks! Do you know of any other
commonly used tools that modify build commands in a similar way and
should be considered?

Best,
Luis

-- 
Luis Augenstein * luis.augenstein@xxxxxxxxxxx * +49-152-25275761
TNG Technology Consulting GmbH, Beta-Str. 13, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Dr. Robert Dahlke, Thomas Endres
Aufsichtsratsvorsitzender: Christoph Stock
Sitz: Unterföhring * Amtsgericht München * HRB 135082

Attachment: OpenPGP_0x795C8ACACDDCFB34.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature