Re: [PATCH 14/42] perf record: Add --index option for building index table

[Adrian Hunter]

On 01/02/15 20:06, Jiri Olsa wrote:
> On Thu, Jan 29, 2015 at 05:06:55PM +0900, Namhyung Kim wrote:
>> The new --index option will create indexed data file which can be
>> processed by multiple threads parallelly.  It saves meta event and
>> sample data in separate files and merges them with an index table.
>>
>> To build an index table, it needs to know exact offsets and sizes for
>> each sample data.  However the offset only can be calculated after the
>> feature data is fixed, and to save feature data it needs to access to
>> the sample data because it needs to mark used DSOs for build-id table.
>>
>> So I ended up with reserving 1MB hole for the feature data area and then
>> put sample data and calculated offsets.  Now an indexed perf data file
>> will look like below:
>>
>>         +---------------------+
>>         |     file header     |
>>         |---------------------|
>>         |                     |
>>         |     meta events     |
>>         |                     |
>>         |---------------------|
>>         |     feature data    |
>>         |   (contains index) -+--+
>>         |---------------------|  |
>>         |      ~1MB hole      |  |
>>         |---------------------|  |
>>         |                     |  |
>>         |    sample data[1] <-+--+
>>         |                     |  |
>>         |---------------------|  |
>>         |                     |  |
>>         |    sample data[2] <-|--+
>>         |                     |  |
>>         |---------------------|  |
>>         |         ...         | ...
>>         +---------------------+
> 
> I also dont see how to store it in a nice way under current header layout,
> but how about bump up the header version for this feature? ;-)
> 
> currently it's:
> 
> struct perf_file_header {
>         u64                             magic;
>         u64                             size;
>         u64                             attr_size;
>         struct perf_file_section        attrs;
>         struct perf_file_section        data;
>         /* event_types is ignored */
>         struct perf_file_section        event_types;
>         DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
> };
> 
> 
> - we already store attrs as a FEATURE so we could omit that
> - your patch stores only synthesized data into 'data' section (-1 idx)
>   this could be stored into separate file and get merged with the rest
> - new header version would have 'features' section, so the features
>   position wouldnt depend on the 'data' end as of now and we could
>   easily store after all data is merged:
> 
> struct perf_file_header {
>         u64                             magic;
>         u64                             size;
>         u64                             attr_size;
>         struct perf_file_section        features;
>         DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
> };
> 
> 
> thoughts?

How come the features are being written before the sample data anyway?
I would have expected:
	- write the data (update the index in memory)
	- write the features (including index)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/