[PATCH 00/34] perf clang: Builtin clang and perfhook support

From: Wang Nan
Date: Mon Nov 14 2016 - 23:09:58 EST

Next message: Wang Nan: "[PATCH 25/34] perf clang jit: Export functions for jitted code"
Previous message: Wang Nan: "[PATCH 33/34] perf clang builtin: Define hook helpers by default"
Next in thread: Wang Nan: "[PATCH 02/34] perf record: Fix segfault when running with suid and kptr_restrict is 1"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

This is version 2 of perf builtin clang patch series. Compare to v1,
add an exciting feature: jit compiling perf hook functions. This
features allows script writer report result through BPF map in a
customized way.

At the end of this cover letter lists an example shows how to capture
and report a syscall histogram.

This patchset is based on current perf/core.

In this patchset:

Patch 1 - 4 are bugfixes left in my local tree.
Patch 5 - 7 are preparation in libbpf.
Patch 8 - 9 introduce perf hook to perf.
Patch 10 - 21 are patches support builtin clang. Some of then are already
collected by Arnaldo in tmp.perf/builtin-clang. They are slightly
adjusted in this v2 series.
Patch 22 - 29 add JIT compiling to builting clang and add them to perf hook.
Patch 30 - 34 are easy of use improvements. 1) builtin clang defines macros
and helpers by default using builtin include headers. Default
headers can be turned off by -UBUILTIN_CLANG_DEFAULT_INCLUDE.
2) allow JITted perf hooks access more functions in libc.

Example: build syscall histogram, exclude syscalls issued by perf itself.
Please note following improvements:
1. Don't need define many bpf helpers by hand.
See bpf_map_lookup_elem and bpf_get_current_pid_tgid.
2. See how pid of git is passed to BPF script attached to
raw_syscalls:sys_enter.

$ cat ./count_syscalls.c
typedef unsigned long u64;

#define BPF_MAP_TYPE_HASH 1
#define BPF_MAP_TYPE_ARRAY 2

enum GVAL {
G_perf_pid,
NR_GVALS
};

struct bpf_map_def SEC("maps") GVALS = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(u64),
.max_entries = NR_GVALS,
};

struct bpf_map_def SEC("maps") syscall_counter = {
.type = BPF_MAP_TYPE_HASH,
.key_size = sizeof(u64),
.value_size = sizeof(u64),
.max_entries = 512,
};

SEC("raw_syscalls:sys_enter")
int func(void *ctx)
{
int key = G_perf_pid;
u64 id = *((u64 *)(ctx + 8));
int self_pid = bpf_get_current_pid_tgid() & 0xffffffff;
int *perf_pid = bpf_map_lookup_elem(&GVALS, &key);
u64 *counter;

if (!perf_pid)
return 0;
if (*perf_pid == self_pid)
return 0;
counter = bpf_map_lookup_elem(&syscall_counter, &id);
if (!counter) {
u64 value = 1;
bpf_map_update_elem(&syscall_counter, &id, &value, 0);
return 0;
}
__sync_fetch_and_add(counter, 1);
return 0;
}

SEC("perfhook:record_start")
void record_start(void *ctx)
{
int perf_pid = getpid(), key = G_perf_pid;
printf("Start count, perfpid=%d\n", perf_pid);
jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
}

SEC("perfhook:record_end")
void record_end(void *ctx)
{
u64 key = -1, value;
while (!jit_helper__map_get_next_key(ctx, &syscall_counter, &key, &key)) {
jit_helper__map_lookup_elem(ctx, &syscall_counter, &key, &value);
printf("syscall %ld\tcount: %ld\n", (long)key, (long)value);
}
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;

$ sudo -s
# ulimit -l unlimited
# perf record -e ./count_syscalls.c echo "Haha"
Start count, perfpid=25209
Haha
[ perf record: Woken up 1 times to write data ]
syscall 8 count: 6
syscall 11 count: 1
syscall 4 count: 6
syscall 21 count: 1
syscall 5 count: 3
syscall 231 count: 1
syscall 45 count: 3
syscall 0 count: 24
syscall 257 count: 1
syscall 59 count: 4
syscall 23 count: 9
syscall 78 count: 2
syscall 41 count: 4
syscall 72 count: 8
syscall 10 count: 3
syscall 321 count: 1
syscall 298 count: 7
syscall 16 count: 21
syscall 9 count: 16
syscall 1 count: 114
syscall 12 count: 3
syscall 14 count: 35
syscall 158 count: 1
syscall 2 count: 15
syscall 7 count: 18
syscall 3 count: 11
[ perf record: Captured and wrote 0.011 MB perf.data ]

Eric Leblond (1):
tools lib bpf: fix maps resolution

Wang Nan (33):
perf tools: Fix kernel version error in ubuntu
perf record: Fix segfault when running with suid and kptr_restrict is
1
tools perf: Add missing struct defeinition in probe_event.h
tools lib bpf: Add missing bpf map functions
tools lib bpf: Add private field for bpf_object
tools lib bpf: Retrive bpf_map through offset of bpf_map_def
perf tools: Introduce perf hooks
perf tools: Pass context to perf hook functions
perf llvm: Extract helpers in llvm-utils.c
tools build: Add feature detection for LLVM
tools build: Add feature detection for clang
perf build: Add clang and llvm compile and linking support
perf clang: Add builtin clang support ant test case
perf clang: Use real file system for #include
perf clang: Allow passing CFLAGS to builtin clang
perf clang: Update test case to use real BPF script
perf clang: Support compile IR to BPF object and add testcase
perf clang: Compile BPF script use builtin clang support
perf clang: Pass full path to builtin clang
perf clang: Pass CFLAGS to builtin clang
perf clang jit: Wrap llvm::Module using PerfModule
perf clang jit: Insignt BPF and JIT functions in a Module
perf clang jit: add PerfModule::doJIT to JIT perfhook functions.
perf clang jit: Export functions for jitted code
perf clang jit: Actually JIT and hook in bpf loader
perf clang jit: Collect the lowest address in maps section as map_base
perf clang jit: Access BPF map
perf clang jit: Allow jitted perf hook access BPF maps
perf clang: Link BPF functions declaration into perf
perf clang: Declare BPF functions for BPF scripts automatically
perf clang: Include helpers to BPF scripts
perf clang builtin: Define hook helpers by default
perf clang git: Export getpid() to perf hook

tools/build/feature/Makefile | 18 +
tools/build/feature/test-clang.cpp | 21 ++
tools/build/feature/test-llvm.cpp | 8 +
tools/lib/bpf/bpf.c | 56 +++
tools/lib/bpf/bpf.h | 7 +
tools/lib/bpf/libbpf.c | 177 ++++++---
tools/lib/bpf/libbpf.h | 13 +
tools/perf/Makefile.config | 62 +++-
tools/perf/Makefile.perf | 23 +-
tools/perf/builtin-record.c | 11 +
tools/perf/tests/Build | 4 +-
tools/perf/tests/bpf-script-example.c | 30 +-
tools/perf/tests/bpf-script-test-kbuild.c | 2 +
tools/perf/tests/bpf-script-test-prologue.c | 6 +-
tools/perf/tests/bpf-script-test-relocation.c | 17 +-
tools/perf/tests/builtin-test.c | 13 +
tools/perf/tests/clang.c | 50 +++
tools/perf/tests/llvm.h | 7 +
tools/perf/tests/make | 2 +
tools/perf/tests/perf-hooks.c | 48 +++
tools/perf/tests/tests.h | 4 +
tools/perf/util/Build | 5 +
tools/perf/util/bpf-loader.c | 102 +++++-
tools/perf/util/bpf-loader.h | 19 +
tools/perf/util/c++/Build | 4 +
tools/perf/util/c++/bpf-funcs-str.c | 228 ++++++++++++
tools/perf/util/c++/bpf-helper-str.c | 24 ++
tools/perf/util/c++/clang-bpf-includes.h | 13 +
tools/perf/util/c++/clang-c.h | 63 ++++
tools/perf/util/c++/clang-test.cpp | 99 +++++
tools/perf/util/c++/clang.cpp | 498 ++++++++++++++++++++++++++
tools/perf/util/c++/clang.h | 61 ++++
tools/perf/util/jit-helpers.c | 60 ++++
tools/perf/util/jit-helpers.h | 30 ++
tools/perf/util/llvm-utils.c | 76 +++-
tools/perf/util/llvm-utils.h | 15 +-
tools/perf/util/perf-hooks-list.h | 3 +
tools/perf/util/perf-hooks.c | 88 +++++
tools/perf/util/perf-hooks.h | 39 ++
tools/perf/util/probe-event.h | 2 +
tools/perf/util/symbol.c | 2 +-
tools/perf/util/util-cxx.h | 26 ++
tools/perf/util/util.c | 55 ++-
43 files changed, 1991 insertions(+), 100 deletions(-)
create mode 100644 tools/build/feature/test-clang.cpp
create mode 100644 tools/build/feature/test-llvm.cpp
create mode 100644 tools/perf/tests/clang.c
create mode 100644 tools/perf/tests/perf-hooks.c
create mode 100644 tools/perf/util/c++/Build
create mode 100644 tools/perf/util/c++/bpf-funcs-str.c
create mode 100644 tools/perf/util/c++/bpf-helper-str.c
create mode 100644 tools/perf/util/c++/clang-bpf-includes.h
create mode 100644 tools/perf/util/c++/clang-c.h
create mode 100644 tools/perf/util/c++/clang-test.cpp
create mode 100644 tools/perf/util/c++/clang.cpp
create mode 100644 tools/perf/util/c++/clang.h
create mode 100644 tools/perf/util/jit-helpers.c
create mode 100644 tools/perf/util/jit-helpers.h
create mode 100644 tools/perf/util/perf-hooks-list.h
create mode 100644 tools/perf/util/perf-hooks.c
create mode 100644 tools/perf/util/perf-hooks.h
create mode 100644 tools/perf/util/util-cxx.h

Signed-off-by: Wang Nan <wangnan0@xxxxxxxxxx>
Cc: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
Cc: Alexei Starovoitov <ast@xxxxxx>
Cc: He Kuang <hekuang@xxxxxxxxxx>
Cc: Jiri Olsa <jolsa@xxxxxxxxxx>
Cc: Zefan Li <lizefan@xxxxxxxxxx>
Cc: pi3orama@xxxxxxx

--
2.10.1

Next message: Wang Nan: "[PATCH 25/34] perf clang jit: Export functions for jitted code"
Previous message: Wang Nan: "[PATCH 33/34] perf clang builtin: Define hook helpers by default"
Next in thread: Wang Nan: "[PATCH 02/34] perf record: Fix segfault when running with suid and kptr_restrict is 1"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]