[GIT PULL 00/37] perf/core improvements and fixes

[Arnaldo Carvalho de Melo]

Hi Ingo,

	Please consider pulling, more to come soon,

- Arnaldo

Build and test stats at the end of the message.

The following changes since commit e9c848928abf4cb60601e9ae7d336f0333c98bca:

  Merge tag 'perf-c2c-for-mingo-20161021' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-10-22 10:22:05 +0200)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20161024

for you to fetch changes up to 04b553ad7dc347eabd3cb4705932272453175a80:

  perf coresight: Removing miscellaneous debug output (2016-10-24 11:07:47 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

New features:

- Dynamicly change verbosity level by pressing 'V' in the 'perf top/report'
  hists TUI browser (Alexis Berlemont)

- Implement 'perf trace --delay' in the same fashion as in 'perf record --delay',
  to skip sampling workload initialization events (Alexis Berlemont)

- Make vendor named events case insensitive in 'perf list', i.e.
  'perf list LONGEST_LAT' works just the same as  'perf list longest_lat' (Andi Kleen)

- Show instruction bytes and lenght in 'perf script' for Intel PT and BTS (Andi Kleen, Adrian Hunter)

   E.g:

    % perf record -e intel_pt// foo
    % perf script --itrace=i0ns -F ip,insn,insnlen
     ffffffff8101232f ilen: 5 insn: 0f 1f 44 00 00
     ffffffff81012334 ilen: 1 insn: 5b
     ffffffff81012335 ilen: 1 insn: 5d
     ffffffff81012336 ilen: 1 insn: c3
     ffffffff810123e3 ilen: 1 insn: 5b
     ffffffff810123e4 ilen: 2 insn: 41 5c
     ffffffff810123e6 ilen: 1 insn: 5d
     ffffffff810123e7 ilen: 1 insn: c3
     ffffffff810124a6 ilen: 2 insn: 31 c0
     ffffffff810124a8 ilen: 9 insn: 41 83 bc 24 a8 01 00 00 01
     ffffffff810124b1 ilen: 2 insn: 75 87

- Allow enabling the perf_event_attr.branch_type attribute member: (Andi Kleen)

  perf record -e sched:sched_switch,cpu/cpu-cycles,branch_type=any/ ...

- Add unwinding support for jitdump (Stefano Sanfilippo)

Fixes:

- Use raw_syscall:sys_enter timestamp in 'perf trace' (Arnaldo Carvalho de Melo)

Infrastructure:

- Allow jitdump to be built without libdwarf (Maciej Debski)

- Sync x86's syscall table tools/ copy (Arnaldo Carvalho de Melo)

- Fixes to avoid calling die() in library fuctions already propagating other
  errors (Arnaldo Carvalho de Melo)

- Improvements to allow libtraceevent to be properly installed in distro
  packages (Jiri Olsa)

- Removing coresight miscellaneous debug output (Mathieu Poirier)

- Cache align the 'perf bench futex' worker struct (Sebastian Andrzej Siewior)

Documentation:

- Minor improvements on the documentation of event parameters (Andi Kleen)

- Add jitdump format specification document (Stephane Eranian)

Spelling fixes:

- Fix typo "No enough" to "Not enough" (Alexander Alemayhu)

Signed-off-by: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>

----------------------------------------------------------------
Adrian Hunter (1):
      perf intel-pt/bts: Tidy instruction buffer size usage

Alexander Alemayhu (1):
      perf tools: Fix typo "No enough" to "Not enough"

Alexis Berlemont (2):
      perf hists browser: Dynamically change verbosity level
      perf trace: Implement --delay

Andi Kleen (6):
      perf intel-pt/bts: Report instruction bytes and length in sample
      perf script: Support insn and insnlen
      perf record: Improve documentation of event parameters
      perf tools: Implement branch_type event parameter
      perf pmu: Only print Using CPUID message once
      perf list: Make vendor event matching case insensitive

Arnaldo Carvalho de Melo (8):
      perf tools: Sync copy of x86's syscall table
      perf jit: Avoid returning garbage for a ret variable
      perf jit: Add NT_GNU_BUILD_ID definition for older distros
      perf bench mem: Move boilerplate memory allocation to the infrastructure
      perf tools: Normalize sq_quote_argv() error reporting
      perf tools: Use normal error reporting when processing PERF_RECORD_READ events
      perf trace: Remove thread_trace->exit_time
      perf trace: Use the syscall raw_syscalls:sys_enter timestamp

Jiri Olsa (8):
      tools lib traceevent: Add install_headers target
      tools lib traceevent: Add do_install_mkdir Makefile function
      tools lib traceevent: Rename LIB_FILE to LIB_TARGET
      tools lib traceevent: Add version for traceevent shared object
      tools lib: Add for_each_clear_bit macro
      perf report: Move captured info to generic header info
      perf header: Display missing features
      perf header: Display feature name on write failure

Maciej Debski (1):
      perf jit: Enable jitdump support without dwarf

Mathieu Poirier (1):
      perf coresight: Removing miscellaneous debug output

Sebastian Andrzej Siewior (1):
      perf bench futex: Cache align the worker struct

Stefano Sanfilippo (5):
      perf jit: Make perf skip unknown records
      perf jit: Do not assume pgoff is zero
      perf jit: Add unwinding support
      perf jit: Generate .eh_frame/.eh_frame_hdr in DSO
      perf jit: Check JITHEADER_VERSION

Stephane Eranian (3):
      perf jit: Improve error messages from JVMTI
      perf jit: Remove unecessary padding in jitdump file
      perf jit: Add jitdump format specification document

 tools/include/asm-generic/bitops.h                 |   1 +
 tools/include/asm-generic/bitops/__ffz.h           |  12 ++
 tools/include/asm-generic/bitops/find.h            |  28 ++++
 tools/include/linux/bitops.h                       |   5 +
 tools/lib/find_bit.c                               |  25 +++
 tools/lib/traceevent/Makefile                      |  40 +++--
 tools/perf/Documentation/jitdump-specification.txt | 170 +++++++++++++++++++++
 tools/perf/Documentation/perf-record.txt           |   9 +-
 tools/perf/Documentation/perf-script.txt           |   6 +-
 tools/perf/Documentation/perf-trace.txt            |   5 +
 tools/perf/MANIFEST                                |   1 +
 tools/perf/Makefile.config                         |   2 +-
 tools/perf/arch/arm/util/cs-etm.c                  |   2 -
 tools/perf/arch/x86/entry/syscalls/syscall_64.tbl  |   4 +-
 tools/perf/bench/futex-hash.c                      |   5 +-
 tools/perf/bench/mem-functions.c                   |  77 ++++------
 tools/perf/builtin-report.c                        |  12 +-
 tools/perf/builtin-script.c                        |  24 ++-
 tools/perf/builtin-trace.c                         |  19 ++-
 tools/perf/jvmti/jvmti_agent.c                     |  38 +----
 tools/perf/jvmti/libjvmti.c                        |  39 +++--
 tools/perf/tests/backward-ring-buffer.c            |   2 +-
 tools/perf/tests/bpf.c                             |   2 +-
 tools/perf/ui/browsers/hists.c                     |   5 +-
 tools/perf/util/Build                              |   2 +-
 tools/perf/util/bpf-loader.c                       |  14 +-
 tools/perf/util/event.h                            |   3 +
 tools/perf/util/evsel.c                            |   9 ++
 tools/perf/util/evsel.h                            |   2 +
 tools/perf/util/genelf.c                           | 113 +++++++++++++-
 tools/perf/util/genelf.h                           |   5 +-
 tools/perf/util/header.c                           |  19 ++-
 tools/perf/util/intel-bts.c                        |   9 +-
 .../perf/util/intel-pt-decoder/intel-pt-decoder.c  |   2 +
 .../perf/util/intel-pt-decoder/intel-pt-decoder.h  |   1 +
 .../util/intel-pt-decoder/intel-pt-insn-decoder.c  |  13 +-
 .../util/intel-pt-decoder/intel-pt-insn-decoder.h  |   6 +-
 tools/perf/util/intel-pt-decoder/intel-pt-log.c    |   4 +-
 tools/perf/util/intel-pt.c                         |  19 ++-
 tools/perf/util/jitdump.c                          |  82 ++++++++--
 tools/perf/util/jitdump.h                          |  12 ++
 tools/perf/util/llvm-utils.c                       |   2 +-
 tools/perf/util/map.c                              |  17 ++-
 tools/perf/util/parse-branch-options.c             |  85 ++++++-----
 tools/perf/util/parse-branch-options.h             |   3 +-
 tools/perf/util/parse-events.c                     |  15 +-
 tools/perf/util/pmu.c                              |  10 +-
 tools/perf/util/quote.c                            |   2 +-
 tools/perf/util/session.c                          |  10 --
 tools/perf/util/string.c                           |  21 ++-
 tools/perf/util/unwind-libunwind-local.c           |   4 +-
 tools/perf/util/util.h                             |   1 +
 tools/perf/util/values.c                           |  81 +++++++---
 tools/perf/util/values.h                           |   4 +-
 54 files changed, 830 insertions(+), 273 deletions(-)
 create mode 100644 tools/include/asm-generic/bitops/__ffz.h
 create mode 100644 tools/perf/Documentation/jitdump-specification.txt

  # time dm
     1 alpine:3.4: Ok
     2 android-ndk:r12b-arm: Ok
     3 archlinux:latest: Ok
     4 centos:5: Ok
     5 centos:6: Ok
     6 centos:7: Ok
     7 debian:7: Ok
     8 debian:8: Ok
     9 debian:experimental: Ok
    10 fedora:20: Ok
    11 fedora:21: Ok
    12 fedora:22: Ok
    13 fedora:23: Ok
    14 fedora:24: Ok
    15 fedora:24-x-ARC-uClibc: Ok
    16 fedora:rawhide: Ok
    17 mageia:5: Ok
    18 opensuse:13.2: Ok
    19 opensuse:42.1: Ok
    20 opensuse:tumbleweed: Ok
    21 ubuntu:12.04.5: Ok
    22 ubuntu:14.04: Ok
    23 ubuntu:14.04.4: Ok
    24 ubuntu:15.10: Ok
    25 ubuntu:16.04: Ok
    26 ubuntu:16.04-x-arm: Ok
    27 ubuntu:16.04-x-arm64: Ok
    28 ubuntu:16.04-x-powerpc: Ok
    29 ubuntu:16.04-x-powerpc64: Ok
    30 ubuntu:16.04-x-powerpc64el: Ok
    31 ubuntu:16.04-x-s390: Ok
    32 ubuntu:16.10: Ok
  
  real	46m1.737s
  user	0m2.200s
  sys	0m2.735s
  #

   # perf test
   1: vmlinux symtab matches kallsyms                          : Ok
   2: detect openat syscall event                              : Ok
   3: detect openat syscall event on all cpus                  : Ok
   4: read samples using the mmap interface                    : Ok
   5: parse events tests                                       : Ok
   6: Validate PERF_RECORD_* events & perf_sample fields       : Ok
   7: Test perf pmu format parsing                             : Ok
   8: Test dso data read                                       : Ok
   9: Test dso data cache                                      : Ok
  10: Test dso data reopen                                     : Ok
  11: roundtrip evsel->name check                              : Ok
  12: Check parsing of sched tracepoints fields                : Ok
  13: Generate and check syscalls:sys_enter_openat event fields: Ok
  14: struct perf_event_attr setup                             : Ok
  15: Test matching and linking multiple hists                 : Ok
  16: Try 'import perf' in python, checking link problems      : Ok
  17: Test breakpoint overflow signal handler                  : Ok
  18: Test breakpoint overflow sampling                        : Ok
  19: Test number of exit event of a simple workload           : Ok
  20: Test software clock events have valid period values      : Ok
  21: Test object code reading                                 : Ok
  22: Test sample parsing                                      : Ok
  23: Test using a dummy software event to keep tracking       : Ok
  24: Test parsing with no sample_id_all bit set               : Ok
  25: Test filtering hist entries                              : Ok
  26: Test mmap thread lookup                                  : Ok
  27: Test thread mg sharing                                   : Ok
  28: Test output sorting of hist entries                      : Ok
  29: Test cumulation of child hist entries                    : Ok
  30: Test tracking with sched_switch                          : Ok
  31: Filter fds with revents mask in a fdarray                : Ok
  32: Add fd to a fdarray, making it autogrow                  : Ok
  33: Test kmod_path__parse function                           : Ok
  34: Test thread map                                          : Ok
  35: Test LLVM searching and compiling                        :
  35.1: Basic BPF llvm compiling test                          : Ok
  35.2: Test kbuild searching                                  : Ok
  35.3: Compile source for BPF prologue generation test        : Ok
  35.4: Compile source for BPF relocation test                 : Ok
  36: Test topology in session                                 : Ok
  37: Test BPF filter                                          :
  37.1: Test basic BPF filtering                               : Ok
  37.2: Test BPF prologue generation                           : Ok
  37.3: Test BPF relocation checker                            : Ok
  38: Test thread map synthesize                               : Ok
  39: Test cpu map synthesize                                  : Ok
  40: Test stat config synthesize                              : Ok
  41: Test stat synthesize                                     : Ok
  42: Test stat round synthesize                               : Ok
  43: Test attr update synthesize                              : Ok
  44: Test events times                                        : Ok
  45: Test backward reading from ring buffer                   : Ok
  46: Test cpu map print                                       : Ok
  47: Test SDT event probing                                   : Ok
  48: Test is_printable_array function                         : Ok
  49: Test bitmap print                                        : Ok
  50: x86 rdpmc test                                           : Ok
  51: Test converting perf time to TSC                         : Ok
  52: Test dwarf unwind                                        : Ok
  53: Test x86 instruction decoder - new instructions          : Ok
  54: Test intel cqm nmi context read                          : Skip
  #

   $ make -C tools/perf build-test
  make: Entering directory '/home/acme/git/linux/tools/perf'
  - tarpkg: ./tests/perf-targz-src-pkg .
                   make_tags_O: make tags 
                  make_debug_O: make DEBUG=1
              make_no_libbpf_O: make NO_LIBBPF=1
                make_no_newt_O: make NO_NEWT=1
             make_no_libperl_O: make NO_LIBPERL=1
                make_minimal_O: make NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1 NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1 NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1 NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1 NO_LIBCRYPTO=1 NO_SDT=1
               make_no_slang_O: make NO_SLANG=1
            make_no_demangle_O: make NO_DEMANGLE=1
           make_no_libpython_O: make NO_LIBPYTHON=1
        make_with_babeltrace_O: make LIBBABELTRACE=1
                  make_no_ui_O: make NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
              make_no_libelf_O: make NO_LIBELF=1
           make_no_backtrace_O: make NO_BACKTRACE=1
             make_util_map_o_O: make util/map.o
           make_no_libunwind_O: make NO_LIBUNWIND=1
             make_no_libnuma_O: make NO_LIBNUMA=1
             make_no_scripts_O: make NO_LIBPYTHON=1 NO_LIBPERL=1
                make_install_O: make install
              make_clean_all_O: make clean all
   make_install_prefix_slash_O: make install prefix=/tmp/krava/
                 make_perf_o_O: make perf.o
                   make_help_O: make help
            make_no_libaudit_O: make NO_LIBAUDIT=1
       make_util_pmu_bison_o_O: make util/pmu-bison.o
                   make_pure_O: make
  make_no_libdw_dwarf_unwind_O: make NO_LIBDW_DWARF_UNWIND=1
            make_no_auxtrace_O: make NO_AUXTRACE=1
         make_install_prefix_O: make install prefix=/tmp/krava
           make_no_libbionic_O: make NO_LIBBIONIC=1
            make_install_bin_O: make install-bin
                make_no_gtk2_O: make NO_GTK2=1
                 make_static_O: make LDFLAGS=-static
                    make_doc_O: make doc
  OK
  make: Leaving directory '/home/acme/git/linux/tools/perf'
  $