Re: [PATCH bpf-next 1/2] bpf: Remove bpf_jit_enable=2 debugging mode

[Christophe Leroy]

Le 20/04/2021 à 05:28, Alexei Starovoitov a écrit :
> On Sat, Apr 17, 2021 at 1:16 AM Christophe Leroy
> <christophe.leroy@xxxxxxxxxx> wrote:
> > Le 16/04/2021 à 01:49, Alexei Starovoitov a écrit :
> > > On Thu, Apr 15, 2021 at 8:41 AM Quentin Monnet <quentin@xxxxxxxxxxxxx> wrote:
> > > > 2021-04-15 16:37 UTC+0200 ~ Daniel Borkmann <daniel@xxxxxxxxxxxxx>
> > > > > On 4/15/21 11:32 AM, Jianlin Lv wrote:
> > > > > > For debugging JITs, dumping the JITed image to kernel log is discouraged,
> > > > > > "bpftool prog dump jited" is much better way to examine JITed dumps.
> > > > > > This patch get rid of the code related to bpf_jit_enable=2 mode and
> > > > > > update the proc handler of bpf_jit_enable, also added auxiliary
> > > > > > information to explain how to use bpf_jit_disasm tool after this change.
> > > > > >
> > > > > > Signed-off-by: Jianlin Lv <Jianlin.Lv@xxxxxxx>
> > > >
> > > > Hello,
> > > >
> > > > For what it's worth, I have already seen people dump the JIT image in
> > > > kernel logs in Qemu VMs running with just a busybox, not for kernel
> > > > development, but in a context where buiding/using bpftool was not
> > > > possible.
> > >
> > > If building/using bpftool is not possible then majority of selftests won't
> > > be exercised. I don't think such environment is suitable for any kind
> > > of bpf development. Much so for JIT debugging.
> > > While bpf_jit_enable=2 is nothing but the debugging tool for JIT developers.
> > > I'd rather nuke that code instead of carrying it from kernel to kernel.
> >
> > When I implemented JIT for PPC32, it was extremely helpfull.
> >
> > As far as I understand, for the time being bpftool is not usable in my environment because it
> > doesn't support cross compilation when the target's endianess differs from the building host
> > endianess, see discussion at
> > https://lore.kernel.org/bpf/21e66a09-514f-f426-b9e2-13baab0b938b@xxxxxxxxxx/
> >
> > That's right that selftests can't be exercised because they don't build.
> >
> > The question might be candid as I didn't investigate much about the replacement of "bpf_jit_enable=2
> > debugging mode" by bpftool, how do we use bpftool exactly for that ? Especially when using the BPF
> > test module ?
>
> the kernel developers can add any amount of printk and dumps to debug
> their code,
> but such debugging aid should not be part of the production kernel.
> That sysctl was two things at once: debugging tool for kernel devs and
> introspection for users.
> bpftool jit dump solves the 2nd part. It provides JIT introspection to users.
> Debugging of the kernel can be done with any amount of auxiliary code
> including calling print_hex_dump() during jiting.

I finally managed to cross compile bpftool with libbpf, libopcodes, readline, ncurses, libcap, libz 
and all needed stuff. Was not easy but I made it.

Now, how do I use it ?

Let say I want to dump the jitted code generated from a call to 'tcpdump'. How do I do that with 
'bpftool prog dump jited' ?

I thought by calling this line I would then get programs dumped in a way or another just like when 
setting 'bpf_jit_enable=2', but calling that line just provides me some bpftool help text.

By the way, I would be nice to have a kernel OPTION that selects all OPTIONS required for building 
bpftool. Because you discover them one by one at every build failure. I had to had CONFIG_IPV6, 
CONFIG_DEBUG_BTF, CONFIG_CGROUPS, ... If there could be an option like "Build a 'bpftool' ready 
kernel" that selected all those, it would be great.

Christophe