Re: [PATCH 10/48] perf tools: Introduce bpf-output event
From: Arnaldo Carvalho de Melo
Date: Wed Feb 24 2016 - 08:36:48 EST
Em Wed, Feb 24, 2016 at 09:58:34AM +0800, Wangnan (F) escreveu:
>
>
> On 2016/2/24 1:45, Arnaldo Carvalho de Melo wrote:
> >Em Mon, Feb 22, 2016 at 09:10:37AM +0000, Wang Nan escreveu:
> >>Commit a43eec304259a6c637f4014a6d4767159b6a3aa3 (bpf: introduce
> >>bpf_perf_event_output() helper) add a helper to enable BPF program
> >>output data to perf ring buffer through a new type of perf event
> >>PERF_COUNT_SW_BPF_OUTPUT. This patch enable perf to create perf
> >>event of that type. Now perf user can use following cmdline to
> >>receive output data from BPF programs:
> >>
> >> # ./perf record -a -e bpf-output/no-inherit,name=evt/ \
> >> -e ./test_bpf_output.c/map:channel.event=evt/ ls /
> >> # ./perf script
> >> perf 1560 [004] 347747.086295: evt: ffffffff811fd201 sys_write ...
> >> perf 1560 [004] 347747.086300: evt: ffffffff811fd201 sys_write ...
> >> perf 1560 [004] 347747.086315: evt: ffffffff811fd201 sys_write ...
> >> ...
> >>
> >>Test result:
> >> # cat ./test_bpf_output.c
> >> /************************ BEGIN **************************/
> >> #include <uapi/linux/bpf.h>
> >> struct bpf_map_def {
> >> unsigned int type;
> >> unsigned int key_size;
> >> unsigned int value_size;
> >> unsigned int max_entries;
> >> };
> >>
> >> #define SEC(NAME) __attribute__((section(NAME), used))
> >> static u64 (*ktime_get_ns)(void) =
> >> (void *)BPF_FUNC_ktime_get_ns;
> >> static int (*trace_printk)(const char *fmt, int fmt_size, ...) =
> >> (void *)BPF_FUNC_trace_printk;
> >> static int (*get_smp_processor_id)(void) =
> >> (void *)BPF_FUNC_get_smp_processor_id;
> >> static int (*perf_event_output)(void *, struct bpf_map_def *, int, void *, unsigned long) =
> >> (void *)BPF_FUNC_perf_event_output;
> >>
> >> struct bpf_map_def SEC("maps") channel = {
> >> .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
> >> .key_size = sizeof(int),
> >> .value_size = sizeof(u32),
> >> .max_entries = __NR_CPUS__,
> >> };
> >>
> >> SEC("func_write=sys_write")
> >> int func_write(void *ctx)
> >> {
> >> struct {
> >> u64 ktime;
> >> int cpuid;
> >> } __attribute__((packed)) output_data;
> >> char error_data[] = "Error: failed to output: %d\n";
> >>
> >> output_data.cpuid = get_smp_processor_id();
> >> output_data.ktime = ktime_get_ns();
> >> int err = perf_event_output(ctx, &channel, get_smp_processor_id(),
> >> &output_data, sizeof(output_data));
> >> if (err)
> >> trace_printk(error_data, sizeof(error_data), err);
> >> return 0;
> >> }
> >> char _license[] SEC("license") = "GPL";
> >> int _version SEC("version") = LINUX_VERSION_CODE;
> >> /************************ END ***************************/
> >>
> >> # ./perf record -a -e bpf-output/no-inherit,name=evt/ \
> >> -e ./test_bpf_output.c/map:channel.event=evt/ ls /
> >> # ./perf script | grep ls
> >> ls 2242 [003] 347851.557563: evt: ffffffff811fd201 sys_write ...
> >> ls 2242 [003] 347851.557571: evt: ffffffff811fd201 sys_write ...
> >So, there is something strange here:
> >
> > if (unlikely(event->oncpu != smp_processor_id()))
> > return -EOPNOTSUPP;
> >
> >This is where I am hitting, with:
> >
> >[acme@jouet linux]$ uname -r
> >4.5.0-rc4
> >
> > int err = perf_event_output(ctx, &channel, get_smp_processor_id(),
> > &output_data, sizeof(output_data));
> > if (err)
> > trace_printk(error_data, sizeof(error_data), err);
> >
> >And then:
> >
> >[root@jouet bpf]# tail /sys/kernel/debug/tracing/trace
> > perf-13040 [003] d... 12062.807729: : Error: failed to output: -95
> > perf-13040 [003] d... 12062.807731: : Error: failed to output: -95
> > perf-13040 [003] d... 12062.807732: : Error: failed to output: -95
> > perf-13040 [003] d... 12062.807735: : Error: failed to output: -95
> > perf-13040 [003] d... 12062.807737: : Error: failed to output: -95
> > perf-13040 [003] d... 12062.807744: : Error: failed to output: -95
> > gnome-terminal--3091 [001] d... 12062.807773: : Error: failed to output: -95
> > gnome-terminal--3091 [001] d... 12062.807784: : Error: failed to output: -95
> > gmain-2830 [002] d... 12062.811791: : Error: failed to output: -95
> > gmain-2830 [002] d... 12062.811810: : Error: failed to output: -95
> >[root@jouet bpf]#
> >
> >Ideas? AFK for a while, will continue investigating.
>
> I also noticed this output, but didn't digg into it because all events
> I concerned is okay. I'll look into this today.
>
> >This already was submitted to Ingo, BTW.
> >
> >I used, as in the changeset comment tests:
> >
> >perf record -a -e bpf-output/no-inherit,name=evt/ -e ./test_bpf_output.c/map:channel.event=evt/ ls /
> >
> >And perf script told me:
> >
> >[root@jouet bpf]# perf script | tail
> > perf 13040 [003] 12062.708337: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708339: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708340: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708341: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708343: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708344: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708346: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708347: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708348: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> > perf 13040 [003] 12062.708350: evt: ffffffff81234eb1 sys_write (/lib/modules/4.5.0-rc4/build/vmlinux)
> >[root@jouet bpf]#
> >
> >Wonder where that /lib/modules/4.5.0-rc4/build/vmlinux came from...
> >
> >[root@jouet bpf]# perf script | cut -d'(' -f2 | sort | uniq -c
> > 1141 /lib/modules/4.5.0-rc4/build/vmlinux)
>
> It's a standard directory for perf searching vmlinux. Isn't it?
Nah, that was me being confused by 'perf script's output, it looked like
what was enclosed in () right after the sys_write was a parameter for
that function (sys_write), when in fact it is the DSO where sys_write is
in, duh.
- Arnaldo
> tools/perf/util/symbol.c:
>
> static const char * const vmlinux_paths_upd[] = {
> "/boot/vmlinux-%s",
> "/usr/lib/debug/boot/vmlinux-%s",
> "/lib/modules/%s/build/vmlinux",
> "/usr/lib/debug/lib/modules/%s/vmlinux",
> "/usr/lib/debug/boot/vmlinux-%s.debug"
> };
>
> So what's your problem?
>
> Thank you.