Re: [perf] perf test and probe fails sometimes on 4.9

[Pintu Kumar]

Hi,

** Changed the subject now, since these issues are related to general
perf commands.

Following are the issues:

1) perf probe --add schedule - FAILED
output:
schedule is out of .text, skip it.
  Error: Failed to add events.

what is the issue here?
Sometimes it pass and sometimes it fails...
Similar is the case of 'perf inject' as well.

2) perf test - 1 FAILURE
37.1: Test basic BPF filtering                               : FAILED!
37.2: Test BPF prologue generation                           : Skip
37.3: Test BPF relocation checker                            : Skip

bpf: config program 'func=SyS_epoll_wait'
symbol:SyS_epoll_wait file:(null) line:0 offset:0 return:0 lazy:(null)
bpf: config 'func=SyS_epoll_wait' is ok
Looking at the vmlinux_path (8 entries long)
Using /boot/vmlinux for symbols
Could not open debuginfo. Try to use symbols.
SyS_epoll_wait is out of .text, skip it.
bpf_probe: failed to convert perf probe eventsFailed to add events
selected by BPF
test child finished with -1
---- end ----
Test BPF filter subtest 0: FAILED!

Looks like both 1,2 are related.
Since, CONFIG_DEBUG_INFO is not enabled, I manually copied the vmlinux
to /boot/ folder.

-------
Some more info:

Kernel build dir is set to /lib/modules/4.9.20-sc-amd-x86-64/build
set env: KBUILD_DIR=/lib/modules/4.9.20-sc-amd-x86-64/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem
/usr/lib/gcc/x86_64-linux-gnu/5/include -I./arch/x86/include
-I./arch/x86/include/generated/uapi -I./arch/x86/include/generated
-I./include -I./arch/x86/include/uapi -I./include/uapi
-I./include/generated/uapi -include ./include/linux/kconfig.h
set env: NR_CPUS=8
set env: LINUX_VERSION_CODE=0x40914
set env: CLANG_EXEC=/usr/bin/clang
set env: CLANG_OPTIONS=-xc
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem
/usr/lib/gcc/x86_64-linux-gnu/5/include -I./arch/x86/include
-I./arch/x86/include/generated/uapi -I./arch/x86/include/generated
-I./include -I./arch/x86/include/uapi -I./include/uapi
-I./include/generated/uapi -include ./include/linux/kconfig.h
set env: WORKING_DIR=/lib/modules/4.9.20-sc-amd-x86-64/build


If you have any clue about these failure please help me.


Thanks,
Pintu


On Wed, Jan 24, 2018 at 8:23 PM, Pintu Kumar <pintu.ping@xxxxxxxxx> wrote:
> Hi,
>
> Thanks for your help.
> Yes it was a sub version issue.
>
> Earlier, while building the kernel I left the SUBLEVEL as blank.
>
> make -j8 bindeb-pkg SUBLEVEL=20
>
> After passing the correct sublevel now the test is working.
>
> But still following are failing:
>
> 16: Try 'import perf' in python, checking link problems      : FAILED!
> 37.2: Test BPF prologue generation                           : FAILED!
>
>
> This is the error I get:
>
> 16: Try 'import perf' in python, checking link problems      :
> --- start ---
> test child forked, pid 7637
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ImportError: No module named perf
> test child finished with -1
> ---- end ----
> Try 'import perf' in python, checking link problems: FAILED!
>
>
> Looking at the vmlinux_path (8 entries long)
> symsrc__init: cannot get elf header.
> Failed to find the path for kernel: Invalid ELF file
> bpf_probe: failed to convert perf probe eventsFailed to add events
> selected by BPF
> test child finished with -1
> ---- end ----
> Test BPF filter subtest 1: FAILED!
>
>
>
> Thanks,
> Pintu
>
>
> On Wed, Jan 24, 2018 at 6:39 AM, Wangnan (F) <wangnan0@xxxxxxxxxx> wrote:
>>
>>
>> On 2018/1/23 20:37, Pintu Kumar wrote:
>>>
>>> Hi All,
>>>
>>> I am verifying all perf tests on Ubuntu-16 x86-64 platform using the
>>> kernel version 4.9.20.
>>> I have installed several others packages including: clang, llvm
>>>
>>> But, when I run 'perf test' I get some FAILURE.
>>> Specially, 'perf test LLVM' is failing.
>>>
>>> Please check the below error logs:
>>>
>>> ----------------------------------------------------
>>> # perf test LLVM
>>> 35: Test LLVM searching and compiling                        :
>>> 35.1: Basic BPF llvm compiling test                          : FAILED!
>>> 35.2: Test kbuild searching                                  : Skip
>>> 35.3: Compile source for BPF prologue generation test        : Skip
>>> 35.4: Compile source for BPF relocation test                 : Skip
>>> ----------------------------------------------------
>>>
>>> When I run with -v I get this:
>>>
>>> -----------------------------------------------------
>>> # perf test -v LLVM
>>> 35: Test LLVM searching and compiling                        :
>>> 35.1: Basic BPF llvm compiling test                          :
>>> --- start ---
>>> test child forked, pid 3304
>>> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64'
>>> WARNING:        unable to get correct kernel building directory.
>>> Hint:   Set correct kbuild directory using 'kbuild-dir' option in [llvm]
>>>          section of ~/.perfconfig or set it to "" to suppress kbuild
>>>          detection.
>>>
>>> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64'
>>>
>>>
>>> int _version SEC("version") = LINUX_VERSION_CODE;
>>> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS
>>> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS
>>> $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign
>>> -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o
>>> -
>>> libbpf: 2129190-4a0 doesn't provide kernel version
>>> Failed to parse test case 'Basic BPF llvm compiling test'
>>> test child finished with -1
>>> ---- end ----
>>> Test LLVM searching and compiling subtest 0: FAILED!
>>> -----------------------------------------------------
>>>
>>>
>>> Basic BPF LLVM compiling test is failing.
>>> It reports that bpf could not able to read the kernel version from uname.
>>> Is it because of this that 'perf test LLVM' is failing ?
>>>
>>> My uname says:
>>> 4.9-xxxx-amd-x86-64
>>
>>
>> 'perf test LLVM' requires a well formed uname string (%d.%d.%d). Please see
>> tools/perf/fetch_kernel_version . If your distro uses another method to
>> report
>> kernel version number (from your uname string, sublevel can't be
>> determined),
>> please help us improve that function like what we have done for ubuntu
>> (checking /proc/version_signature).
>>
>>
>>
>>> My kernel version is: 4.9.20 (from Makefile)
>>>
>>> So, I dont think there is any problem with kernel version.
>>>
>>> If any body have come across this problem please let me know how to
>>> resolve this issue.
>>>
>>>
>>>
>>> Thank You!
>>>
>>> Regards,
>>> Pintu
>>
>>
>>