Re: [PATCH v2 0/3] Assume libbpf 1.0+

[Jiri Olsa]

On Sat, Mar 11, 2023 at 05:00:01PM -0800, Ian Rogers wrote:
> On Fri, Mar 10, 2023 at 12:22 PM Andrii Nakryiko
> <andrii.nakryiko@xxxxxxxxx> wrote:
> >
> > On Thu, Mar 9, 2023 at 7:26 PM Ian Rogers <irogers@xxxxxxxxxx> wrote:
> > >
> > > On Thu, Mar 9, 2023 at 9:25 AM Andrii Nakryiko
> > > <andrii.nakryiko@xxxxxxxxx> wrote:
> > > >
> > > > On Wed, Mar 8, 2023 at 11:58 PM Guilherme Amadio <amadio@xxxxxxxxxx> wrote:
> > > > >
> > > > > On Wed, Mar 08, 2023 at 06:13:34PM -0800, Ian Rogers wrote:
> > > > > > On Thu, Jan 19, 2023 at 1:13 PM Jiri Olsa <olsajiri@xxxxxxxxx> wrote:
> > > > > > >
> > > > > > > On Thu, Jan 19, 2023 at 02:41:12PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > > > > > Em Thu, Jan 19, 2023 at 09:11:03AM -0800, Ian Rogers escreveu:
> > > > > > > > > On Sun, Jan 15, 2023 at 5:01 PM Ian Rogers <irogers@xxxxxxxxxx> wrote:
> > > > > > > > > > libbpf 1.0 was a major change in API. Perf has partially supported
> > > > > > > > > > older libbpf's but an implementation may be:
> > > > > > > > > > ..
> > > > > > > > > >        pr_err("%s: not support, update libbpf\n", __func__);
> > > > > > > > > >        return -ENOTSUP;
> > > > > > > > > > ..
> > > > > > > > > >
> > > > > > > > > > Rather than build a binary that would fail at runtime it is
> > > > > > > > > > preferrential just to build libbpf statically and link against
> > > > > > > > > > that. The static version is in the kernel tools tree and newer than
> > > > > > > > > > 1.0.
> > > > > > > > > >
> > > > > > > > > > These patches change the libbpf test to only pass when at least
> > > > > > > > > > version 1.0 is installed, then remove the conditional build and
> > > > > > > > > > feature logic.
> > > > > > > > > >
> > > > > > > > > > The issue is discussed here:
> > > > > > > > > > https://lore.kernel.org/lkml/20230106151320.619514-1-irogers@xxxxxxxxxx/
> > > > > > > > > > perf bpf:
> > > > > > > > > >
> > > > > > > > > > A variant of this fix was added to Linux 6.2 in:
> > > > > > > > > > "perf bpf: Avoid build breakage with libbpf < 0.8.0 + LIBBPF_DYNAMIC=1"
> > > > > > > > > > https://lore.kernel.org/lkml/Y71+eh00Ju7WeEFX@xxxxxxxxxx/
> > > > > > > > > > This change goes further in removing logic that is now no longer
> > > > > > > > > > necessary.
> > > > > > > > > >
> > > > > > > > > > v2. Rebase now that breakage fix patch is in linus/master.
> > > > > > > > >
> > > > > > > > > I missed the:
> > > > > > > > > Acked/Tested-by: Jiri Olsa <jolsa@xxxxxxxxxx>
> > > > > > > > > I believe we are waiting for package maintainer input.
> > > > > > > >
> > > > > > > > Yes, as fedora:37 still is at libbpf 0.8.0 :-\
> > > > > > >
> > > > > > > rawhide (f38) is already on 1.1.0 ... I'll check how bad it'd be to move
> > > > > > > f37 to 1.x, but I had to do bulk update of like 10 other dependent packages
> > > > > > > for f38, so not sure how bad it'd be for f37
> > > > > > >
> > > > > > > jirka
> > > > > >
> > > > > > +Guilherme
> > > > > >
> > > > > > We were looking for maintainer input on these changes, but there is no
> > > > > > update in over a month. Here is the original lore link:
> > > > > > https://lore.kernel.org/lkml/CAP-5=fVUgc8xtBzGi66YRUxZHyXvW2kiMjGz39dywaLxrO4Hpg@xxxxxxxxxxxxxx/
> > > > > > Should these changes land in perf-tools-next targeting Linux 6.4?
> > > > >
> > > > > Gentoo has libbpf-1.1 already available, so requiring >libbpf-1.0 is not
> > > > > a problem. We (Gentoo) just need to make sure to stabilize libbpf-1.x before
> > > > > stabilizing newer versions of perf, as the stable libbpf is 0.8.1 at the moment.
> > > > >
> > > >
> > > > libbpf v0.8 is basically all the 1.0 APIs, except by default 1.0
> > > > semantics is not enforced, unless libbpf_set_strict_mode() is enabled.
> > > >
> > > > So, if 0.8 is a restriction, perf can stay on 0.8, use all the same
> > > > APIs that are in 1.0 (except newer one added later, but I'm not sure
> > > > perf needs any of the newer additions), and just stick to setting
> > > > libbpf_set_strict_mode() unconditionally.
> > >
> > > Thanks Andrii,
> > >
> >
> > Full disclosure, I'm totally supporting the switch to v1.0+, just
> > trying to be helpful here from the standpoint of 0.x vs 1.x libbpf
> > transition. See below. I believe you can keep 0.8+ dependency and drop
> > all the legacy code completely.
> >
> > But just take it as an information, and feel free to do whatever you
> > think is best with it.
> >
> > > The default perf build is to build against tools/lib/bpf and
> > > statically link libbpf in. This means by default we have the latest
> > > libbpf 1.2. If any perf code has a dependency on 0.8 (we don't support
> > > earlier) we need to #ifdef for it. Currently we have 7 feature tests
> > > for libbpf, but perhaps there is some cruft that's carried forward.
> > > The features are:
> > >  - btf__load_from_kernel_by_id
> >
> > v0.5 API
> >
> > >  - bpf_prog_load
> > >  - bpf_object__next_program
> > >  - bpf_object__next_map
> >
> > all three are v0.6 APIs
> >
> > >  - bpf_program__set_insns
> >
> > v0.8 API
> >
> > >  - btf__raw_data
> > >  - bpf_map_create
> >
> > both v0.6 API
> >
> > >
> > > The not present implementations look like:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/util/bpf-loader.c?h=perf-tools#n36
> > > ```
> > > int bpf_program__set_insns(struct bpf_program *prog __maybe_unused,
> > >   struct bpf_insn *new_insns __maybe_unused, size_t new_insn_cnt __maybe_unused)
> > > {
> > > pr_err("%s: not support, update libbpf\n", __func__);
> > > return -ENOTSUP;
> > > }
> > >
> > > int libbpf_register_prog_handler(const char *sec __maybe_unused,
> > >                                  enum bpf_prog_type prog_type __maybe_unused,
> > >                                  enum bpf_attach_type exp_attach_type
> > > __maybe_unused,
> > >                                  const struct libbpf_prog_handler_opts
> > > *opts __maybe_unused)
> > > {
> > > pr_err("%s: not support, update libbpf\n", __func__);
> > > return -ENOTSUP;
> > > }
> > > ```
> >
> > both are v0.8 APIs
> >
> > > This will basically mean that while you dynamically linked with libbpf
> > > 0.8 you are in all likelihood not going to get proper BPF support.
> > > These changes up the version requirement to 1.0 and get rid entirely
> > > of the feature tests - so no runtime failing implementations. If the
> >
> > 100% supportive on upgrade and dropping feature checks. My point is
> > that you don't need those feature checks with v0.8+ requirement.
> >
> > The only difference between staying on v0.8+ vs going all the way to
> > v1.0+ would be that you have to keep libbpf_set_strict() call. In
> > v1.0+ it's a noop, so could be dropped.
> >
> > > build determines at build time libbpf 1.0+ isn't present then it still
> > > executes, switching from dynamic libbpf to the default static libbpf
> > > that is at 1.2. As mentioned in this thread, distributions like Debian
> > > use the default static linking of libbpf.
> > >
> >
> > oh, that's nice, good to know
> >
> > > I'm not keen to hold on to the feature tests for the complexity that
> > > they hold and their needlessly (as you can always statically link)
> > > broken at runtime behavior. We could but my opinion is, let's not :-)
> >
> > I've been consistently advocating for static linking with libbpf, so
> > 100% support this.
> 
> Right, so I think we should move forward with these patches. We also

+1

jirka

> have had non-API changes to support libbpf 1.0+ like:
> https://lore.kernel.org/lkml/20221103045437.163510-4-irogers@xxxxxxxxxx/
> and I'm worried about the state of these with libbpf 0.8.
> 
> Thanks,
> Ian
> 
> > >
> > > Thanks,
> > > Ian
> > >
> > > > > Best regards,
> > > > > -Guilherme
> > > > >