[RFC PATCH bpf-next 00/12] bpf: Introduce static-defined tracing probe for BPF

[Xu Kuohai]

From: Xu Kuohai <xukuohai@xxxxxxxxxx>

This series introduces static-defined tracing probes for BPF programs.
BPF SDT (static-defined tracing) works similarly to USDT. User defines
probes in the BPF source code. The probes are built into NOP instructions
in the ELF. At runtime, when an observer is attached, the NOP instruction
is patched to a CALL instruction to the observer prog trampoline.

Unlike USDT, BPF SDT requires explicit macros to generate the function
prototype BTF for each probe. This allows the verifier to validate the
probe sites against the declared types, and observer programs can be
attached similarly to normal tracing programs using the function prototype
information.

A probe with two arguments in the target program can be declared and defined
like:

    BPF_SDT_DECLARE2(my_trace, int, int);

    SEC("xdp")
    int xdp_prog(struct xdp_md *ctx)
    {
        int len = ctx->data_end - ctx->data;
        int ret = XDP_DROP;
        ...
        BPF_SDT_PROBE2(my_trace, len, ret);
        ...
    }

An observer would be like:

    SEC("bpf_sdt")
    int BPF_PROG(observer_prog, int len, int ret)
    {
        bpf_printk("len=%d ret=%d\n", len, ret);
        return 0;
    }

The target program and probe site for the observer program should be set
at load time via bpf_program__set_attach_target() manually, since program
names are not unique and program IDs are allocated at runtime - there is
no static way to identify the target program.

For BPF_SDT_DECLARE2(my_trace, int, int) and BPF_SDT_PROBE2(my_trace, len, ret)
macros, the compiler produces:

  [code section, e.g. xdp]
      goto +0                  // NOP, patched to CALL at attach time

  [.bpf_sdt_notes section]
      ___sdt_jt_my_trace:      // symbol marking this entry's boundary
      .quad  0b                // 8 bytes: offset of the NOP in the code
                               // section (resolved by the linker via
                               // R_BPF_64_ABS64 relocation)
      r1 = %[arg1_reg]         // 8 bytes per argument: BPF move insn
                               // whose src_reg field encodes the BPF
                               // register holding each probe argument
      r2 = %[arg2_reg]

      ....

  [.BTF section]
      FUNC_PROTO (int, int) -> void   // from BPF_SDT_DECLARE2 stub
      DECL_TAG "bpf_sdt:my_trace:2"   // keyed by name + nargs

The .BTF and .bpf_sdt_notes sections are parsed by libbpf and ultimately
converted to an insn_array map per bpf program. Each map slot records the
instruction offset, jited address, FUNC_PROTO BTF ID, and argument register
for each probe site.

When an observer prog is loaded, the probe is resolved against the target
program's SDT map using the target fd and probe name passed by the user.

For attach, the existing bpf trampoline is reused with per-argument register
mappings declared by the SDT macro.

Xu Kuohai (12):
  libbpf: Prepare bpf SDT probe section for the linker
  libbpf: Introduce bpf SDT probe macros
  libbpf: Add bpf_sdt_notes section parser
  bpf: Create insn_array map for bpf SDT probe
  bpf: Collect SDT probe BTF IDs from BTF decl tags
  bpf: Add type check for SDT probe site
  bpf: Record probe name in SDT map
  libbpf: Add libbpf support to load SDT observer program
  bpf: Add kernel support to load SDT observer program
  bpf: Support attach and detach for SDT observer program
  bpf, x86: Add JIT support SDT for probe
  selftests/bpf: Add tests for bpf SDT probe

 arch/x86/net/bpf_jit_comp.c                   |  38 +-
 include/linux/bpf.h                           |  26 +-
 include/linux/bpf_types.h                     |   1 +
 include/linux/bpf_verifier.h                  |   3 +
 include/linux/filter.h                        |   1 +
 include/uapi/linux/bpf.h                      |  21 +-
 kernel/bpf/bpf_insn_array.c                   |  64 ++-
 kernel/bpf/cfg.c                              |   3 +
 kernel/bpf/core.c                             |   5 +
 kernel/bpf/fixups.c                           |   4 +
 kernel/bpf/liveness.c                         |  24 +-
 kernel/bpf/syscall.c                          | 163 +++++-
 kernel/bpf/trampoline.c                       |   4 +
 kernel/bpf/verifier.c                         | 174 +++++-
 tools/include/uapi/linux/bpf.h                |  21 +-
 tools/lib/bpf/Makefile                        |   2 +-
 tools/lib/bpf/bpf.c                           |  10 +-
 tools/lib/bpf/bpf.h                           |   9 +-
 tools/lib/bpf/bpf_sdt.h                       | 346 ++++++++++++
 tools/lib/bpf/libbpf.c                        | 513 +++++++++++++++++-
 tools/lib/bpf/libbpf_internal.h               |   1 +
 tools/lib/bpf/linker.c                        |   9 +
 .../selftests/bpf/prog_tests/test_bpf_sdt.c   | 151 ++++++
 .../selftests/bpf/progs/bpf_sdt_observer.c    |  30 +
 .../selftests/bpf/progs/bpf_sdt_target.c      |  34 ++
 25 files changed, 1622 insertions(+), 35 deletions(-)
 create mode 100644 tools/lib/bpf/bpf_sdt.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_observer.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_target.c

-- 
2.47.3