Re: [PATCH 14/42] perf record: Add --index option for building index table

[Jiri Olsa]

On Thu, Jan 29, 2015 at 05:06:55PM +0900, Namhyung Kim wrote:
> The new --index option will create indexed data file which can be
> processed by multiple threads parallelly.  It saves meta event and
> sample data in separate files and merges them with an index table.
> 
> To build an index table, it needs to know exact offsets and sizes for
> each sample data.  However the offset only can be calculated after the
> feature data is fixed, and to save feature data it needs to access to
> the sample data because it needs to mark used DSOs for build-id table.
> 
> So I ended up with reserving 1MB hole for the feature data area and then
> put sample data and calculated offsets.  Now an indexed perf data file
> will look like below:
> 
>         +---------------------+
>         |     file header     |
>         |---------------------|
>         |                     |
>         |     meta events     |
>         |                     |
>         |---------------------|
>         |     feature data    |
>         |   (contains index) -+--+
>         |---------------------|  |
>         |      ~1MB hole      |  |
>         |---------------------|  |
>         |                     |  |
>         |    sample data[1] <-+--+
>         |                     |  |
>         |---------------------|  |
>         |                     |  |
>         |    sample data[2] <-|--+
>         |                     |  |
>         |---------------------|  |
>         |         ...         | ...
>         +---------------------+

I also dont see how to store it in a nice way under current header layout,
but how about bump up the header version for this feature? ;-)

currently it's:

struct perf_file_header {
        u64                             magic;
        u64                             size;
        u64                             attr_size;
        struct perf_file_section        attrs;
        struct perf_file_section        data;
        /* event_types is ignored */
        struct perf_file_section        event_types;
        DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
};


- we already store attrs as a FEATURE so we could omit that
- your patch stores only synthesized data into 'data' section (-1 idx)
  this could be stored into separate file and get merged with the rest
- new header version would have 'features' section, so the features
  position wouldnt depend on the 'data' end as of now and we could
  easily store after all data is merged:

struct perf_file_header {
        u64                             magic;
        u64                             size;
        u64                             attr_size;
        struct perf_file_section        features;
        DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
};


thoughts?
jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/