Re: [PATCH v4 00/10] selftests/bpf: Tolerate partial builds across kernel configs

[Alexei Starovoitov]

On Fri, Apr 10, 2026 at 6:57 AM Ricardo B. Marlière <rbm@xxxxxxxx> wrote:
>
> Hi Alan,
>
> On Fri Apr 10, 2026 at 7:06 AM -03, Alan Maguire wrote:
> > On 06/04/2026 19:22, Ricardo B. Marlière wrote:
> >> Currently the BPF selftests can only be built by using the minimum kernel
> >> configuration defined in tools/testing/selftests/bpf/config*. This poses a
> >> problem in distribution kernels that may have some of the flags disabled or
> >> set as module. For example, we have been running the tests regularly in
> >> openSUSE Tumbleweed [1] [2] but to work around this fact we created a
> >> special package [3] that build the tests against an auxiliary vmlinux with
> >> the BPF Kconfig. We keep a list of known issues that may happen due to,
> >> amongst other things, configuration mismatches [4] [5].
> >>
> >> The maintenance of this package is far from ideal, especially for
> >> enterprise kernels. The goal of this series is to enable the common usecase
> >> of running the following in any system:
> >>
> >> ```sh
> >> make -C tools/testing/selftests install \
> >>         SKIP_TARGETS= \
> >>         TARGETS=bpf \
> >>         O=/lib/modules/$(uname -r)/build
> >> ```
> >>
> >> As an example, the following script targeting a minimal config can be used
> >> for testing:
> >>
> >> ```sh
> >> make defconfig
> >> scripts/config --file .config \
> >>                --enable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT \
> >>                --enable DEBUG_INFO_BTF \
> >>                --enable BPF_SYSCALL \
> >>                --enable BPF_JIT
> >> make olddefconfig
> >> make -j$(nproc)
> >> make -j$(nproc) -C tools/testing/selftests SKIP_TARGETS= TARGETS=bpf install
> >> ```
> >>
> >> This produces a test_progs binary with 579 subtests, against the total of
> >> 708. Many of them will still fail or be skipped at runtime due to lack of
> >> symbols, but at least there will be a clear way of building the tests.
> >>
> >
> > hi Ricardo, I've been looking at this series and one concern I had was if
> > tolerating failures to build with incomplete config would create problems
> > in detecting failures when the config is complete, i.e. in BPF CI. So for
> > example if say a BPF skeleton didn't compile when it really should due to
> > an introduced bug, we wouldn't want to drive on in all cases.
>
> Agreed.
>
> >
> > However, from what I can see, CI held up well with these changes in place [1].
> >
> > That said it might be nice to preserve a "strict" mode which fails when
> > we hit compilation issues etc, failing as we do today; I couldn't find any existing
> > general kselftest concept that fits with that but it would be valuable I think
> > if possible. Perhaps we could default to non-strict mode when a make -C is initiated
> > or something like that, and continue to operate strictly when building directly
> > in tools/testing/selftests/bpf?
> >
> > I ran the changes through the usual workflow I use - build bpf selftests at
> > tools/testing/selftests/bpf level and run ./test_progs ; no regressions were
> > observed, so feel free to add for the series
> >
> > Tested-by: Alan Maguire <alan.maguire@xxxxxxxxxx>
>
> Thanks for testing!
>
> >
> > There are some comments on sashiko.dev [2] that would be worth looking through
> > too, in particular the concern about O= and M= mentioned for patch 1.
>
> I will send v5 with those addressed and possibly a strict-mode switch.

and strict should be the default.

Even sashiko doesn't like this "ignore all errors" mode:

https://sashiko.dev/#/patchset/20260406-selftests-bpf_misconfig-v4-0-9914f50efdf7%40suse.com