Re: [PATCHv6 0/5] perf record: Add support to store data in directory

[Namhyung Kim]

On Mon, Mar 11, 2019 at 12:30 AM Jiri Olsa <jolsa@xxxxxxxxxx> wrote:
>
> hi,
> this patchset adds the --dir option to record command (and all
> the other record command that overload cmd_record) that allows
> the data to be stored in directory with multiple data files.
>
> It's next step for multiple threads implementation in record.
> It's now possible to make directory data via --dir option, like:
>
>   $ perf record --dir perf bench sched messaging
>   $ ls -l perf.data
>   total 344
>   -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0
>   -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1
>   -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2
>   -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3
>   -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4
>   -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5
>   -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6
>   -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7
>   -rw-------. 1 jolsa jolsa  8832 Jan 20 22:26 header
>
> There's a data file created for every cpu and it's storing
> data for those cpu maps. The report command will read it
> transparently, sort it and display as single file data.
>
> There's new DIR_FORMAT feature to describe directory data
> layout/format. In future we can describe different data files
> layout according to special needs.
>
> It's possible to transform directory data into standard
> perf.data file via simple inject command:
>
>   $ perf inject -o perf.data.file -i perf.data
>
> The old perf fails over the directory data with following message:
>   $ perf report
>   incompatible file format (rerun with -v to learn more)
>
> I'm now testing the record threads support, so I'd like to
> have some agreement on the directory data support before.
>
> v6 changes:
>   - rebased to latest Arnaldo's perf/core
>   - some of the patches already taken
>   - adding comments to __perf_session__process_dir_events
>
> v5 changes:
>   - rebased to latest Arnaldo's perf/core
>
> v4 changes:
>   - some of the patches already taken
>   - removing up perf.data directory on perf_data__open
>     error path
>
> v3 changes:
>   - add rm_rf_perf_data to safely remove file/directory perf data
>   - allocation fix in perf_data__create_dir
>
> v2 changes:
>   - rm_rf changes are already accepted with requested changes
>   - updated doc/man plus adding perf.data-directory-format.txt
>     to describe directory format/layout
>   - the --switch-output options now works over directory data
>   - data rollback is not part of this patchset, updated my TODO though ;-)
>   - added --output-dir option to combine -o and --dir
>   - added DIR_FORMAT feature to describe directory data
>   - disabling directory output for aio for now
>
> It's also available in here:
>   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
>   perf/dir
>

It seems I'm too late, but anyway for the series

Acked-by: Namhyung Kim <namhyung@xxxxxxxxxx>

Thanks for the work!
Namhyung



> Jiri Olsa (5):
>       perf session: Add __perf_session__process_dir_events function
>       perf session: Add path to reader object
>       perf record: Add --dir option to store data in directory
>       perf record: Add --output-dir option to store data in directory
>       perf record: Describe perf.data directory format
>
>  tools/lib/subcmd/parse-options.h                        |   4 ++++
>  tools/perf/Documentation/perf-record.txt                |   6 ++++++
>  tools/perf/Documentation/perf.data-directory-format.txt |  54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tools/perf/builtin-record.c                             |  93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
>  tools/perf/util/mmap.h                                  |  23 ++++++++++++-----------
>  tools/perf/util/session.c                               | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
>  6 files changed, 272 insertions(+), 22 deletions(-)
>  create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt