Re: perf test LLVM & clang 6 failing
From: Yonghong Song
Date: Mon Nov 27 2017 - 14:12:43 EST
On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
[ +Yonghong ]
+ Josh
On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.
Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)
So, if I go to the cset just before:
commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf <jpoimboe@xxxxxxxxxx>
Date: Wed Sep 20 16:24:33 2017 -0500
x86/asm: Fix inline asm call constraints for Clang
---
'perf test LLVM' works again:
[root@jouet ~]# perf test LLVM
37: LLVM search and compile :
37.1: Basic BPF llvm compile : Ok
37.2: kbuild searching : Ok
37.3: Compile source for BPF prologue generation : Ok
37.4: Compile source for BPF relocation : Ok
[root@jouet ~]#
I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.
This patch introduced a module level inline assembly.
...
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -132,4 +132,15 @@
/* For C file, we already have NOKPROBE_SYMBOL macro */
#endif
+#ifndef __ASSEMBLY__
+/*
+ * This output constraint should be used for any inline asm which has a
"call"
+ * instruction. Otherwise the asm may be inserted before the frame pointer
+ * gets set up by the containing function. If you forget to do this,
objtool
+ * may print a "call without frame pointer save/setup" warning.
+ */
+register unsigned int __asm_call_sp asm("esp");
+#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
+#endif
...
This will cause "clang ... -target bpf ..." failure since 4.0 does not
have bpf inline asm support and 6.0 does not recognize the register 'esp'.
'perf test BPF' breaks for another reason, needs another investigation,
maybe you guys can spot something changed recently in that area:
libbpf: failed to create map (name: 'flip_table'): Invalid argument
libbpf: failed to load object '[basic_bpf_test]'
Not sure whether this is the root cause or not, but the below section name
SEC("func=SyS_epoll_wait")
is not a good one.
If it is used to represent the program name in the kernel, the kernel
will reject it with invalid argument since it contains "=".
Complete logs up to this error are below, I'll go read Yonghong message
to see if I can get the 'perf test LLVM' case fixed.
- Arnaldo
[root@jouet ~]# perf test BPF
39: BPF filter :
39.1: Basic BPF filtering : FAILED!
39.2: BPF pinning : Skip
39.3: BPF prologue generation : Skip
39.4: BPF relocation checker : Skip
[root@jouet ~]# perf test -v BPF
39: BPF filter :
39.1: Basic BPF filtering :
--- start ---
test child forked, pid 3541
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/lib64/ccache/clang
set env: CLANG_OPTIONS=-xc
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
* bpf-script-example.c
* Test basic LLVM building
*/
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2
static void *(*bpf_map_lookup_elem)(void *map, void *key) =
(void *) BPF_FUNC_map_lookup_elem;
static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
(void *) BPF_FUNC_map_update_elem;
struct bpf_map_def {
unsigned int type;
unsigned int key_size;
unsigned int value_size;
unsigned int max_entries;
};
#define SEC(NAME) __attribute__((section(NAME), used))
struct bpf_map_def SEC("maps") flip_table = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(int),
.max_entries = 1,
};
SEC("func=SyS_epoll_wait")
int bpf_func__SyS_epoll_wait(void *ctx)
{
int ind =0;
int *flag = bpf_map_lookup_elem(&flip_table, &ind);
int new_flag;
if (!flag)
return 0;
/* flip flag and store back */
new_flag = !*flag;
bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
return new_flag;
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o -
libbpf: loading object '[basic_bpf_test]' from buffer
libbpf: section .strtab, size 120, link 0, flags 0, type=3
libbpf: section .text, size 0, link 0, flags 6, type=1
libbpf: section func=SyS_epoll_wait, size 200, link 0, flags 6, type=1
libbpf: found program func=SyS_epoll_wait
libbpf: section .relfunc=SyS_epoll_wait, size 32, link 8, flags 0, type=9
libbpf: section maps, size 16, link 0, flags 3, type=1
libbpf: section license, size 4, link 0, flags 3, type=1
libbpf: license of [basic_bpf_test] is GPL
libbpf: section version, size 4, link 0, flags 3, type=1
libbpf: kernel version of [basic_bpf_test] is 40e00
libbpf: section .symtab, size 168, link 1, flags 0, type=2
libbpf: maps in [basic_bpf_test]: 1 maps in 16 bytes
libbpf: map 0 is "flip_table"
libbpf: collecting relocating info for: 'func=SyS_epoll_wait'
libbpf: relocation: insn_idx=4
libbpf: relocation: find map 0 (flip_table) for insn 4
libbpf: relocation: insn_idx=18
libbpf: relocation: find map 0 (flip_table) for insn 18
bpf: config program 'func=SyS_epoll_wait'
symbol:SyS_epoll_wait file:(null) line:0 offset:0 return:0 lazy:(null)
bpf: config 'func=SyS_epoll_wait' is ok
Looking at the vmlinux_path (8 entries long)
Using /lib/modules/4.14.0+/build/vmlinux for symbols
Open Debuginfo file: /lib/modules/4.14.0+/build/vmlinux
Try to find probe point from debuginfo.
Matched function: SyS_epoll_wait [2e5a9be]
found inline addr: 0xffffffff812af256
Probe point found: compat_SyS_epoll_pwait+150
found inline addr: 0xffffffff812af037
Probe point found: SyS_epoll_pwait+135
found inline addr: 0xffffffff812aeed0
Probe point found: SyS_epoll_wait+0
Found 3 probe_trace_events.
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Writing event: p:perf_bpf_probe/func _text+2814550
Writing event: p:perf_bpf_probe/func_1 _text+2814007
Writing event: p:perf_bpf_probe/func_2 _text+2813648
libbpf: failed to create map (name: 'flip_table'): Invalid argument
libbpf: failed to load object '[basic_bpf_test]'
bpf: load objects failed
Failed to add events selected by BPF