[PATCH v16 00/15] perf tools: Support overwritable ring buffer

[Wang Nan]

This patch set enables daemonized perf recording by utilizing
overwritable backward ring buffer. With this feature one can
put perf background, and dump ring buffer records by a SIGUSR2
when he/she find something unusual. For example, following
command record system calls, schedule events and samples on cpu cycles
continously:

 # perf record -g -e cycles -e raw_syscalls:*/call-graph=no/ \
                  -e sched:sched_switch/call-graph=no/ \
                  --switch-output --overwrite -a

Then by sending SIGUSR2 to perf when lagging is happen, we get multiple
perf.data output, each of them correspond a abnormal event, and the data
size is reasonable:

 # ls -l ./perf.data*
 -rw------- 1 root root 5122165 May 13 23:51 ./perf.data.2016051323511683
 -rw------- 1 root root 5135093 May 13 23:51 ./perf.data.2016051323512107
 -rw------- 1 root root 5135213 May 13 23:51 ./perf.data.2016051323512215
 -rw------- 1 root root 5135157 May 13 23:51 ./perf.data.2016051323512387

v1 -> v2: Totally redesign: drop the principle of 'channal', use
          auxiliary evlist instead. Fix missing documentation.

v2 -> v3: Rename perf_evlist__toggle_paused() to perf_evlist__pause/resume.

v3 -> v4: Update commit message to describe auxiliary evlist more clearly.

v4 -> v5: Reorder commits, ensure '--overwrite' works right after perf
          support the option.
          Add test cases for auxiliary evlist.
          Avoid bug if main evlist is empty.

v5 -> v6: Improve filter pollfd related code.

v6 -> v7: Rebase to newest perf/core.

v7 -> v8: Unmap mmaps from parent and children in
          perf_evlist__munmap_filtered(), hide more detail of aux evlist.
          Add --tail-synthesize, do synthesize at the end of perf.data.

v8 -> v9: Beautify code of test case, make patch set more granular,
          improve documentation.

v9 -> v10: Make patch set more granular: extract preparation code to
           patch 1-3.

v10 -> v11: Rebase to newest perf/core: solve conflicts caused by commit
            e5cadb93d08 ("perf evlist: Rename for_each() macros to
            for_each_entry()").

v11 -> v12: Improve 'perf test backward': skip this test on old kernel,
            resolve conflicts.

v12 -> v13: Drop evsel->overwrite, use evsel->attr.write_backward instead.

v13 -> v14: Follow Jiri Olsa's suggestion: Improve commit message,
            add OVERWRITE_EVT_NOTREADY state, stop the state machine if
	    overwrite_evlist is not generated.

v14 -> v15: Totally redesign: drop auxiliary evlist, use one evlist to
            contain normal and backward ring buffers (add evlist->backward_mmap).
            Build new API and helpers for it.

v15 -> v16: Follow Jiri Olsa's suggestion:
            1. Move backward ring buffer state machine to evlist.
	    2. Dynamically allocate backward mmap, so backward_mmap == NULL
	       means there's no backward ring buffer.
	       Stop the state machine when there's no backward ring buffer.
	    3. Rename: _output2 to _output_backward.
	    4. Patch rearrangement.
	    5. Update record__pick_pc(): read from backward_mmap if normal
               mmap is empty.

Arnaldo Carvalho de Melo (1):
  perf tools: Drop redundant evsel->overwrite indicator

Wang Nan (14):
  tools lib fd array: Allow associating a pointer cookie with each entry
  perf tools: Update perf evlist mmap related APIs and helpers
  perf tools: Decouple record__mmap_read() and evlist.
  perf tools: Record mmap cookie into fdarray private field
  perf tools: Extract common code in mmap failure processing
  perf tools: Introduce backward_mmap array for evlist
  perf tools: Map backward events to backward_mmap
  perf tools: Drop evlist->backward
  perf tools: Setup backward mmap state machine
  perf record: Read from overwritable ring buffer
  perf tools: Make perf_evlist__{pause,resume} internal helpers
  perf tools: Enable overwrite settings
  perf tools: Don't warn about out of order event if write_backward is
    used
  perf tools: Add --tail-synthesize option

 tools/lib/api/fd/array.h                 |   1 +
 tools/perf/Documentation/perf-record.txt |  22 +++
 tools/perf/builtin-record.c              | 113 ++++++++++---
 tools/perf/perf.h                        |   2 +
 tools/perf/tests/backward-ring-buffer.c  |  14 +-
 tools/perf/util/evlist.c                 | 270 ++++++++++++++++++++++---------
 tools/perf/util/evlist.h                 |  48 +++++-
 tools/perf/util/evsel.c                  |  16 +-
 tools/perf/util/evsel.h                  |   3 +-
 tools/perf/util/parse-events.c           |  20 ++-
 tools/perf/util/parse-events.h           |   2 +
 tools/perf/util/parse-events.l           |   2 +
 tools/perf/util/session.c                |  22 ++-
 13 files changed, 411 insertions(+), 124 deletions(-)

Signed-off-by: Wang Nan <wangnan0@xxxxxxxxxx>
Cc: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
Cc: He Kuang <hekuang@xxxxxxxxxx>
Cc: Jiri Olsa <jolsa@xxxxxxxxxx>
Cc: Masami Hiramatsu <mhiramat@xxxxxxxxxx>
Cc: Namhyung Kim <namhyung@xxxxxxxxxx>
Cc: Zefan Li <lizefan@xxxxxxxxxx>
Cc: Nilay Vaish <nilayvaish@xxxxxxxxx>
Cc: pi3orama@xxxxxxx
-- 
1.8.3.4