Re: [PATCH v4 00/10] selftests/bpf: Tolerate partial builds across kernel configs

[Alan Maguire]

On 06/04/2026 19:22, Ricardo B. Marlière wrote:
> Currently the BPF selftests can only be built by using the minimum kernel
> configuration defined in tools/testing/selftests/bpf/config*. This poses a
> problem in distribution kernels that may have some of the flags disabled or
> set as module. For example, we have been running the tests regularly in
> openSUSE Tumbleweed [1] [2] but to work around this fact we created a
> special package [3] that build the tests against an auxiliary vmlinux with
> the BPF Kconfig. We keep a list of known issues that may happen due to,
> amongst other things, configuration mismatches [4] [5].
> 
> The maintenance of this package is far from ideal, especially for
> enterprise kernels. The goal of this series is to enable the common usecase
> of running the following in any system:
> 
> ```sh
> make -C tools/testing/selftests install \
>         SKIP_TARGETS= \
>         TARGETS=bpf \
>         O=/lib/modules/$(uname -r)/build
> ```
> 
> As an example, the following script targeting a minimal config can be used
> for testing:
> 
> ```sh
> make defconfig
> scripts/config --file .config \
>                --enable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT \
>                --enable DEBUG_INFO_BTF \
>                --enable BPF_SYSCALL \
>                --enable BPF_JIT
> make olddefconfig
> make -j$(nproc)
> make -j$(nproc) -C tools/testing/selftests SKIP_TARGETS= TARGETS=bpf install
> ```
> 
> This produces a test_progs binary with 579 subtests, against the total of
> 708. Many of them will still fail or be skipped at runtime due to lack of
> symbols, but at least there will be a clear way of building the tests.
>

hi Ricardo, I've been looking at this series and one concern I had was if
tolerating failures to build with incomplete config would create problems
in detecting failures when the config is complete, i.e. in BPF CI. So for
example if say a BPF skeleton didn't compile when it really should due to 
an introduced bug, we wouldn't want to drive on in all cases.

However, from what I can see, CI held up well with these changes in place [1].

That said it might be nice to preserve a "strict" mode which fails when
we hit compilation issues etc, failing as we do today; I couldn't find any existing 
general kselftest concept that fits with that but it would be valuable I think
if possible. Perhaps we could default to non-strict mode when a make -C is initiated
or something like that, and continue to operate strictly when building directly
in tools/testing/selftests/bpf?

I ran the changes through the usual workflow I use - build bpf selftests at
tools/testing/selftests/bpf level and run ./test_progs ; no regressions were
observed, so feel free to add for the series

Tested-by: Alan Maguire <alan.maguire@xxxxxxxxxx>

There are some comments on sashiko.dev [2] that would be worth looking through
too, in particular the concern about O= and M= mentioned for patch 1.

[1] https://github.com/kernel-patches/bpf/actions/runs/24168181680
[2] https://sashiko.dev/#/patchset/20260406-selftests-bpf_misconfig-v4-0-9914f50efdf7%40suse.com
 > [1]: https://openqa.opensuse.org/tests/5811715
> [2]: https://openqa.opensuse.org/tests/5811730
> [3]: https://src.opensuse.org/rmarliere/kselftests
> [4]: https://github.com/openSUSE/kernel-qe/blob/main/kselftests_known_issues.yaml
> [5]: https://openqa.opensuse.org/tests/5811730/logfile?filename=run_kselftests-config_mismatches.txt
> 
> Assisted-by: {codex,claude}
> Signed-off-by: Ricardo B. Marlière <rbm@xxxxxxxx>
> ---
> Changes in v4:
> - Drop the test_kmods kselftest module flow patch: lib.mk gen_mods_dir
>   invokes $(MAKE) -C $(TEST_GEN_MODS_DIR) without forwarding
>   RESOLVE_BTFIDS, breaking ASAN and GCC BPF CI builds (Makefile.modfinal
>   cannot find resolve_btfids in the kbuild output tree)
> - Link to v3:
>   https://patch.msgid.link/20260406-selftests-bpf_misconfig-v3-0-587a1114263c@xxxxxxxx
> 
> Changes in v3:
> - Split test_kmods patch into two: fix KDIR handling (O= passthrough,
>   EXTRA_CFLAGS/EXTRA_LDFLAGS clearing) and wire into lib.mk via
>   TEST_GEN_MODS_DIR
> - Pass O= through to the kernel module build so artifacts land in the
>   output tree, not the source tree
> - Clear EXTRA_CFLAGS and EXTRA_LDFLAGS when invoking the kernel build to
>   prevent host flags (e.g. -static) leaking into module compilation
> - Replace the bespoke test_kmods pattern rule with lib.mk module
>   infrastructure (TEST_GEN_MODS_DIR); lib.mk now drives build and clean
>   lifecycle
> - Make the .ko copy step resilient: emit SKIP instead of failing when a
>   module is absent
> - Expand the uprobe weak stub comment in bpf_cookie.c to explain why
>   noinline is required
> - Link to v2:
>   https://patch.msgid.link/20260403-selftests-bpf_misconfig-v2-0-f06700380a9d@xxxxxxxx
> 
> Changes in v2:
> - Skip test_kmods build/clean when KDIR directory does not exist
> - Use `Module.symvers` instead of `.config` for in-tree detection
> - Fix skeleton order-only prereqs commit message
> - Guard BTFIDS step when .test.o is absent
> - Add `__weak stack_mprotect()` stubs in `bpf_cookie.c` and `iters.c`
> - Link to v1:
>   https://patch.msgid.link/20260401-selftests-bpf_misconfig-v1-0-3ae42c0af76f@xxxxxxxx
> 
> ---
> Ricardo B. Marlière (10):
>       selftests/bpf: Fix test_kmods KDIR to honor O= and distro kernels
>       selftests/bpf: Tolerate BPF and skeleton generation failures
>       selftests/bpf: Avoid rebuilds when running emit_tests
>       selftests/bpf: Make skeleton headers order-only prerequisites of .test.d
>       selftests/bpf: Tolerate test file compilation failures
>       selftests/bpf: Allow test_progs to link with a partial object set
>       selftests/bpf: Tolerate benchmark build failures
>       selftests/bpf: Provide weak definitions for cross-test functions
>       selftests/bpf: Skip tests whose objects were not built
>       selftests/bpf: Tolerate missing files during install
> 
>  tools/testing/selftests/bpf/Makefile               | 113 ++++++++++++++-------
>  .../testing/selftests/bpf/prog_tests/bpf_cookie.c  |  24 ++++-
>  tools/testing/selftests/bpf/prog_tests/iters.c     |  10 +-
>  tools/testing/selftests/bpf/test_kmods/Makefile    |  15 ++-
>  tools/testing/selftests/bpf/test_progs.c           |   7 +-
>  5 files changed, 120 insertions(+), 49 deletions(-)
> ---
> base-commit: a1aa9ef47c299c5bbc30594d3c2f0589edf908e6
> change-id: 20260401-selftests-bpf_misconfig-4c33ef5c56da
> 
> Best regards,
> --  
> Ricardo B. Marlière <rbm@xxxxxxxx>
> 
>