Re: [PATCH bpf-next] libbpf: Resolve typeless ksyms in the kernel

[Samuel Wu]

On Fri, Oct 16, 2026 at 11:55 AM Andrii Nakryiko
<andrii.nakryiko@xxxxxxxxx> wrote:
>
> On Sun, Oct 11, 2026 at 8:56 AM Samuel Wu <wusamuel@xxxxxxxxxx> wrote:
> >
> > On Sat, Oct 10, 2026 at 12:09 PM Alan Maguire <alan.maguire@xxxxxxxxxx> wrote:
> > >
> > > On 09/10/2026 14:07, Samuel Wu wrote:
> > > > libbpf resolves typeless symbols by parsing /proc/kallsyms, which
> > > > dominates load time for BPF programs with typeless symbols. This mostly
> > > > comes from needing to do thousands of syscalls and a linear scan of
> > > > kallsyms.
> > > >
> > > > Call bpf_kallsyms_lookup_name() from BPF_PROG_TYPE_SYSCALL, such that
> > > > the resolution can be done with a couple of syscalls and a binary
> > > > search. This is a similar approach to what the light skeleton does. The
> > > > fast path passes the name and result through ctx, which requires commit
> > > > ae5ef001aa98 ("bpf: Support variable offsets for syscall PTR_TO_CTX").
> > > >
> > > > Any fast path error (e.g. kernels without ae5ef001aa98) falls back to
> > > > the legacy /proc/kallsyms scan, including strong misses, since the
> > > > kernel doesn't match LTO "<name>.llvm.<hash>" symbols. Weak misses
> > > > default to zero.
> > > >
> > > > Depending on the device, I see ~9-11x loading speedup when resolving a
> > > > BPF program with 11 typeless lock externs.
> > > >   - x86_64 VM, 6.18 + ae5ef001aa98: 147ms -> 17ms
> > > >   - ARM64 phone, 7.1: 626ms -> 55ms
> > > >
> > > > Signed-off-by: Samuel Wu <wusamuel@xxxxxxxxxx>
> > > > ---
> > > >  tools/lib/bpf/libbpf.c | 59 +++++++++++++++++++++++++++++++++++++++++-
> > > >  1 file changed, 58 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> > > > index cb09ded90773..7564ec6bde64 100644
> > > > --- a/tools/lib/bpf/libbpf.c
> > > > +++ b/tools/lib/bpf/libbpf.c
> > > > @@ -9590,6 +9590,58 @@ static int bpf_object__read_kallsyms_file(struct bpf_object *obj)
> > > >       return libbpf_kallsyms_parse(kallsyms_cb, obj);
> > > >  }
> > > >
> > > > +static int bpf_object__resolve_ksyms_in_kernel(struct bpf_object *obj)
> > > > +{
> > > > +     struct extern_desc *ext;
> > > > +     int i, prog_fd, err = 0;
> > > > +     struct {
> > > > +             __u64 addr;
> > > > +             char name[512];
> > > > +     } ctx = {};
> > > > +     LIBBPF_OPTS(bpf_prog_load_opts, opts, .prog_flags = BPF_F_SLEEPABLE);
> > > > +     LIBBPF_OPTS(bpf_test_run_opts, topts, .ctx_in = &ctx, .ctx_size_in = sizeof(ctx));
> > > > +     /* bpf_kallsyms_lookup_name(ctx.name, sizeof(ctx.name), 0, &ctx.addr) */
> > > > +     static const struct bpf_insn insns[] = {
> > > > +             BPF_MOV64_REG(BPF_REG_4, BPF_REG_1),
> > > > +             BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 8),
> > > > +             BPF_MOV64_IMM(BPF_REG_2, sizeof(ctx.name)),
> > > > +             BPF_MOV64_IMM(BPF_REG_3, 0),
> > > > +             BPF_EMIT_CALL(BPF_FUNC_kallsyms_lookup_name),
> > > > +             BPF_EXIT_INSN(),
> > > > +     };
> > > > +
> > > > +     if (obj->gen_loader || obj->token_fd)
> > > > +             return -EOPNOTSUPP;
> > > > +
> > > > +     prog_fd = bpf_prog_load(BPF_PROG_TYPE_SYSCALL, NULL, "GPL",
> > > > +                             insns, ARRAY_SIZE(insns), &opts);
> > > > +     if (prog_fd < 0)
> > > > +             return prog_fd;
> > > > +
> > > > +     for (i = 0; i < obj->nr_extern; i++) {
> > > > +             ext = &obj->externs[i];
> > > > +             if (ext->type != EXT_KSYM || ext->ksym.type_id)
> > > > +                     continue;
> > > > +             if (snprintf(ctx.name, sizeof(ctx.name), "%s", ext->name) >= sizeof(ctx.name)) {
> > > > +                     err = -ENAMETOOLONG;
> > > > +                     break;
> > > > +             }
> > > > +             err = bpf_prog_test_run_opts(prog_fd, &topts) ?: (int)topts.retval;
> > > > +             if (err == -ENOENT && ext->is_weak) {
> > > > +                     err = 0;
> > > > +                     continue;
> > > > +             }
> > > > +             if (err)
> > > > +                     break;
> > > > +             ext->is_set = true;
> > > > +             ext->ksym.addr = ctx.addr;
> > > > +             pr_debug("extern (ksym) '%s': resolved in kernel to 0x%llx\n",
> > > > +                      ext->name, ext->ksym.addr);
> > > > +     }
> > >
> > > Great to see the win in terms of resolution time, but I wonder if we could
> > > speed things up further by resolving multiple symbols at once, since we have
> > > all the externs available (either by dynamically creating your program that
> > > does symbol resolution as a side-effect, or adding a more direct resolution method)?
> > >
> > > A single syscall could provide an even faster speedup, so might be worth a look.
> >
> > I tried resolving multiple symbols in one pass, but unfortunately the
> > impact was negligible and made the code much more complex. I profiled
> > a few numbers, and turns out it's because resolving typeless symbols
> > is down to ~1% of total load time (from ~90%), with the remaining time
> > mostly spent on BTF loading, CORE relocation, and BPF verification.
> >
> > > Another concern would be a lot of production kernels won't have BPF test run
> > > support baked in, so relying on that without fallback would be a problem.
> >
> > Are you referring to BPF_PROG_TEST_RUN when mentioning "BPF test run
> > support", or something else? And totally agreed in that we need a
> > fallback, of which it currently falls back to reading /proc/kallsyms
> > on any error during the new fast path.
> >
>
> Have you considered adding a faster interface to /proc/kallsyms
> instead? That would benefit not just this particular use case, but
> generally lots of other user space applications that have to deal with
> kallsyms (all kinds of symbolizers, tracers, etc).
>
> A while ago I added PROCMAP_QUERY ioctl()-based interface to
> /proc/PID/maps to query process VMAs and I think it turned out quite
> nicely and is super useful in practice. Back then I was hoping we'll
> eventually also add similar (in principle) extensible ioctl()-based
> interface to /proc/kallsyms as well, so that at least fast
> address-to-symbol and symbol name-to-symbol point query could be done
> quickly. And it actually would finally provide a way to provide more
> information per kernel symbol, if necessary (/proc/kallsyms is
> basically set in stone, besides it being text based and thus slow and
> inconvenient for programmatic access).
>
> Can you please look into that, I think it would be a much more generic
> and desirable outcome instead of this somewhat hacky solution of
> calling BPF program to resolve a symbol.
>
> WDYT?

Thanks for the feedback Andrii. An ioctl for kallsyms makes sense. I
agree it would be useful for userspace outside of libbpf, and it would
probably provide a significant speedup vs today's text based parsing.

I'll send a patchset out when I get something working!

> pw-bot: cr
>
>
> > > > +     close(prog_fd);
> > > > +     return err;
> > > > +}
> > > > +
> > > >  static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name,
> > > >                           __u16 kind, struct btf **res_btf,
> > > >                           struct module_btf **res_mod_btf)
> > > > @@ -9863,7 +9915,12 @@ static int bpf_object__resolve_externs(struct bpf_object *obj,
> > > >                       return -EINVAL;
> > > >       }
> > > >       if (need_kallsyms) {
> > > > -             err = bpf_object__read_kallsyms_file(obj);
> > > > +             err = bpf_object__resolve_ksyms_in_kernel(obj);
> > > > +             if (err) {
> > > > +                     pr_debug("failed to resolve ksyms in kernel: %s, falling back to /proc/kallsyms\n",
> > > > +                              errstr(err));
> > > > +                     err = bpf_object__read_kallsyms_file(obj);
> > > > +             }
> > > >               if (err)
> > > >                       return -EINVAL;
> > > >       }
> > >