Re: Intermittent perf build failures

From: Jiri Olsa
Date: Tue Oct 11 2016 - 16:59:33 EST

Next message: Bin Gao: "Re: [PATCH v2] x86/tsc: Set X86_FEATURE_TSC_RELIABLE to skip refined calibration"
Previous message: Ondrej Zary: "Re: [PATCH] scsi: NCR5380: no longer mark irq probing as __init"
In reply to: Laura Abbott: "Intermittent perf build failures"
Next in thread: Jiri Olsa: "Re: Intermittent perf build failures"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Tue, Oct 11, 2016 at 01:43:36PM -0700, Laura Abbott wrote:
> Hi,
>
> While building today's Fedora rawhide kernel, there was a failure
> building perf with -j4 [1]:
>
>
> gcc -Wp,-MD,./.plugin_scsi.o.d,-MT,plugin_scsi.o -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -D"BUILD_STR(s)=#s" -c -o plugin_scsi.o plugin_scsi.c
> ld -r -o plugin_function-in.o plugin_function.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_jbd2.so plugin_jbd2-in.o
> gcc -Wp,-MD,./.plugin_cfg80211.o.d,-MT,plugin_cfg80211.o -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -D"BUILD_STR(s)=#s" -c -o plugin_cfg80211.o plugin_cfg80211.c
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_hrtimer.so plugin_hrtimer-in.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_kmem.so plugin_kmem-in.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_kvm.so plugin_kvm-in.o
> ld -r -o plugin_xen-in.o plugin_xen.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_mac80211.so plugin_mac80211-in.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_sched_switch.so plugin_sched_switch-in.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_function.so plugin_function-in.o
> ld -r -o plugin_cfg80211-in.o plugin_cfg80211.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_xen.so plugin_xen-in.o
> gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -shared -nostartfiles -o plugin_cfg80211.so plugin_cfg80211-in.o
> gcc -Wp,-MD,./.plugin_scsi.o.d,-MT,plugin_scsi.o -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fPIC -I. -I /builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/include '-DPLUGIN_DIR="/usr/lib64/traceevent/plugins"' -D_GNU_SOURCE -D"BUILD_STR(s)=#s" -c -o plugin_scsi.o plugin_scsi.c
> ld -r -o plugin_scsi-in.o plugin_scsi.o
> plugin_scsi.o: file not recognized: File truncated
> make[3]: *** [/builddir/build/BUILD/kernel-4.8.fc26/linux-4.9.0-0.rc0.git6.1.fc26.aarch64/tools/build/Makefile.build:140: plugin_scsi-in.o] Error 1
> make[2]: *** [Makefile:189: plugin_scsi-in.o] Error 2
> make[1]: *** [Makefile.perf:570: libtraceevent_plugins] Error 2
> make: *** [Makefile:109: install-traceevent-plugins] Error 2
>
> The gcc call to compile plugin_scsi.o seems to be invoked twice which would
> account for the ld failing. Unfortunately, this failure seems to be
> intermittent and I've only been able to reproduce it once outside of
> the Fedora build environment so bisection doesn't seem feasible.

hi,
yep, looks like build race.. but -j4 should not be that bad ;-)

some questions:
- how often do you see this?
- is it always plugin_scsi.o failing?
- hows the build invoked?
- could you please attach full build log?

thanks,
jirka

>
> The build invocation for this was changed in
> 0c3b7e42616f ("tools build: Add support for host programs format") so
> I'm suspicious of that commit. I'm reverting that commit for today's
> build but I'm not sure how much that's going to indicate given the
> intermittent nature of the failure.
>
> Any ideas?
>
> Thanks,
> Laura
>
>
> [1] https://kojipkgs.fedoraproject.org//work/tasks/415/16050415/build.log

Next message: Bin Gao: "Re: [PATCH v2] x86/tsc: Set X86_FEATURE_TSC_RELIABLE to skip refined calibration"
Previous message: Ondrej Zary: "Re: [PATCH] scsi: NCR5380: no longer mark irq probing as __init"
In reply to: Laura Abbott: "Intermittent perf build failures"
Next in thread: Jiri Olsa: "Re: Intermittent perf build failures"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]