Re: [RFC PATCH bpf-next 00/12] bpf: Introduce static-defined tracing probe for BPF

[Leon Hwang]

Hi Kuohai,

On 28/6/26 06:51, Xu Kuohai wrote:
> From: Xu Kuohai <xukuohai@xxxxxxxxxx>
> 
> This series introduces static-defined tracing probes for BPF programs.
> BPF SDT (static-defined tracing) works similarly to USDT. User defines


At first glance, the SDT idea looks cool to me.

However, what's your purpose of introducing SDT?

If to provide points in bpf progs to be traced, like tracepoints in
kernel functions, I think subprog+fentry is an alternative approach.
Comparing with SDT, subprog+fentry requires a function call at run time,
instead of a NOP like SDT.

For example,

#define __sink(expr) asm volatile("" : "+g"(expr))

static __noinline void my_trace(int len, int ret)
{
	__sink(len);
	__sink(ret);
}

SEC("xdp")
int xdp_prog(struct xdp_md *ctx)
{
	int len = ctx->data_end - ctx->data;
	int ret = XDP_DROP;
	...
	my_trace(len, ret);
	...
}

'my_trace' can be traced using fentry to inspect the details of 'xdp_prog'.

Furthermore, if users don't want a function call at run time, e.g. they
don't want to call 'my_trace' at run time in production, they can patch
the callsite of 'my_trace' with NOP before loading 'xdp_prog', and drop
the subprog 'my_trace' in their user space application. This elimination
is approachable, since it is used heavily in bpfsnoop [1]. However, this
elimination is not easy to understand. Want me to show more details
about this elimination?

Link:
[1] https://github.com/bpfsnoop/bpfsnoop

Thanks,
Leon

> probes in the BPF source code. The probes are built into NOP instructions
> in the ELF. At runtime, when an observer is attached, the NOP instruction
> is patched to a CALL instruction to the observer prog trampoline.
> 
> Unlike USDT, BPF SDT requires explicit macros to generate the function
> prototype BTF for each probe. This allows the verifier to validate the
> probe sites against the declared types, and observer programs can be
> attached similarly to normal tracing programs using the function prototype
> information.
> 
> A probe with two arguments in the target program can be declared and defined
> like:
> 
>     BPF_SDT_DECLARE2(my_trace, int, int);
> 
>     SEC("xdp")
>     int xdp_prog(struct xdp_md *ctx)
>     {
>         int len = ctx->data_end - ctx->data;
>         int ret = XDP_DROP;
>         ...
>         BPF_SDT_PROBE2(my_trace, len, ret);
>         ...
>     }
> 
> An observer would be like:
> 
>     SEC("bpf_sdt")
>     int BPF_PROG(observer_prog, int len, int ret)
>     {
>         bpf_printk("len=%d ret=%d\n", len, ret);
>         return 0;
>     }
> 
> The target program and probe site for the observer program should be set
> at load time via bpf_program__set_attach_target() manually, since program
> names are not unique and program IDs are allocated at runtime - there is
> no static way to identify the target program.
> 
> For BPF_SDT_DECLARE2(my_trace, int, int) and BPF_SDT_PROBE2(my_trace, len, ret)
> macros, the compiler produces:
> 
>   [code section, e.g. xdp]
>       goto +0                  // NOP, patched to CALL at attach time
> 
>   [.bpf_sdt_notes section]
>       ___sdt_jt_my_trace:      // symbol marking this entry's boundary
>       .quad  0b                // 8 bytes: offset of the NOP in the code
>                                // section (resolved by the linker via
>                                // R_BPF_64_ABS64 relocation)
>       r1 = %[arg1_reg]         // 8 bytes per argument: BPF move insn
>                                // whose src_reg field encodes the BPF
>                                // register holding each probe argument
>       r2 = %[arg2_reg]
> 
>       ....
> 
>   [.BTF section]
>       FUNC_PROTO (int, int) -> void   // from BPF_SDT_DECLARE2 stub
>       DECL_TAG "bpf_sdt:my_trace:2"   // keyed by name + nargs
> 
> The .BTF and .bpf_sdt_notes sections are parsed by libbpf and ultimately
> converted to an insn_array map per bpf program. Each map slot records the
> instruction offset, jited address, FUNC_PROTO BTF ID, and argument register
> for each probe site.
> 
> When an observer prog is loaded, the probe is resolved against the target
> program's SDT map using the target fd and probe name passed by the user.
> 
> For attach, the existing bpf trampoline is reused with per-argument register
> mappings declared by the SDT macro.
> 
> Xu Kuohai (12):
>   libbpf: Prepare bpf SDT probe section for the linker
>   libbpf: Introduce bpf SDT probe macros
>   libbpf: Add bpf_sdt_notes section parser
>   bpf: Create insn_array map for bpf SDT probe
>   bpf: Collect SDT probe BTF IDs from BTF decl tags
>   bpf: Add type check for SDT probe site
>   bpf: Record probe name in SDT map
>   libbpf: Add libbpf support to load SDT observer program
>   bpf: Add kernel support to load SDT observer program
>   bpf: Support attach and detach for SDT observer program
>   bpf, x86: Add JIT support SDT for probe
>   selftests/bpf: Add tests for bpf SDT probe
> 
>  arch/x86/net/bpf_jit_comp.c                   |  38 +-
>  include/linux/bpf.h                           |  26 +-
>  include/linux/bpf_types.h                     |   1 +
>  include/linux/bpf_verifier.h                  |   3 +
>  include/linux/filter.h                        |   1 +
>  include/uapi/linux/bpf.h                      |  21 +-
>  kernel/bpf/bpf_insn_array.c                   |  64 ++-
>  kernel/bpf/cfg.c                              |   3 +
>  kernel/bpf/core.c                             |   5 +
>  kernel/bpf/fixups.c                           |   4 +
>  kernel/bpf/liveness.c                         |  24 +-
>  kernel/bpf/syscall.c                          | 163 +++++-
>  kernel/bpf/trampoline.c                       |   4 +
>  kernel/bpf/verifier.c                         | 174 +++++-
>  tools/include/uapi/linux/bpf.h                |  21 +-
>  tools/lib/bpf/Makefile                        |   2 +-
>  tools/lib/bpf/bpf.c                           |  10 +-
>  tools/lib/bpf/bpf.h                           |   9 +-
>  tools/lib/bpf/bpf_sdt.h                       | 346 ++++++++++++
>  tools/lib/bpf/libbpf.c                        | 513 +++++++++++++++++-
>  tools/lib/bpf/libbpf_internal.h               |   1 +
>  tools/lib/bpf/linker.c                        |   9 +
>  .../selftests/bpf/prog_tests/test_bpf_sdt.c   | 151 ++++++
>  .../selftests/bpf/progs/bpf_sdt_observer.c    |  30 +
>  .../selftests/bpf/progs/bpf_sdt_target.c      |  34 ++
>  25 files changed, 1622 insertions(+), 35 deletions(-)
>  create mode 100644 tools/lib/bpf/bpf_sdt.h
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c
>  create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_observer.c
>  create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_target.c
>