[PATCH v2 0/5] tracing: Add __rel_loc support

[Masami Hiramatsu]

Hello,

Here is the 2nd version of the series of '__rel_loc' relative
dynamic array attribute support. The previous version is here;

https://lore.kernel.org/all/163697159970.131454.2661507704362599471.stgit@devnote2/T/#u

In this version, I added bitmask support, since the bitmask
data is also allocated dynamically. And replace
TEP_FIELD_IS_REL_DYNAMIC with TEP_FIELD_IS_RELATIVE because
this new flag complements the TEP_FIELD_IS_DYNAMIC.

Here are 5 patches, [1/5] is the main and required patch, which
updates the event filter, histogram, and inject for '__rel_loc'.
[2/5] and [3/5] are sample code for '__rel_loc' attribute.
[2/5] adds macros which uses '__rel_loc' instead of '__data_loc'
and [3/5] expands example module of trace event.

[4/5] is updating in-kernel libtraceevent, which is deprecated
but perf-tools still need it. [5/5] is updating perf-tools itself.
I'll send a patch to current libtraceevent soon.

The __rel_loc
-----

The '__data_loc' is used for encoding the dynamic data location on
the trace event record. But '__data_loc' is not useful if the writer
doesn't know the event header (e.g. user event), because it records
the dynamic data offset from the entry of the record, not the field
itself.

This new '__rel_loc' attribute encodes the data location relatively
from the next of the field. For example, when there is a record like
below (the number in the parentheses is the size of fields)

 |header(N)|common(M)|fields(K)|__data_loc(4)|fields(L)|data(G)|

In this case, '__data_loc' field will be

 __data_loc = (G << 16) | (N+M+K+4+L)

If '__rel_loc' is used, this will be

 |header(N)|common(M)|fields(K)|__rel_loc(4)|fields(L)|data(G)|

where

 __rel_loc = (G << 16) | (L)

(Because this case shows L bytes after the '__rel_loc' attribute
 field, if there is no fields after the __rel_loc field, L must be 0.)

This is relatively easy (and no need to consider the kernel header
change) when the event data fields are composed by user who doesn't
know header and common fields.

NOTE: Event Injection doesn't work because of bitmask.

Preparation
----
/ # modprobe trace-events-sample.ko
/ # cd /sys/kernel/tracing/
/sys/kernel/tracing # cat events/sample-trace/foo_rel_loc/format 
name: foo_rel_loc
ID: 1259
format:
	field:unsigned short common_type;	offset:0;	size:2;	signed:0;
	field:unsigned char common_flags;	offset:2;	size:1;	signed:0;
	field:unsigned char common_preempt_count;	offset:3;	size:1;	signed:0;
	field:int common_pid;	offset:4;	size:4;	signed:1;

	field:__rel_loc char foo[];	offset:8;	size:4;	signed:1;
	field:int bar;	offset:12;	size:4;	signed:1;
	field:__rel_loc unsigned long bitmask[];	offset:16;	size:4;	signed:0;

print fmt: "foo_rel_loc %s, %d, %s", __get_rel_str(foo), REC->bar, __get_rel_bitmask(bitmask)


Test Event Histogram
----

/sys/kernel/tracing # echo 'hist:key=foo' >> events/sample-trace/foo_rel_loc/trigger 
/sys/kernel/tracing # cat events/sample-trace/foo_rel_loc/hist 
# event histogram
#
# trigger info: hist:keys=foo:vals=hitcount:sort=hitcount:size=2048 [active]
#

{ foo: Hello __rel_loc                                    } hitcount:         11

Totals:
    Hits: 11
    Entries: 1
    Dropped: 0


Test Event Filter
----
/sys/kernel/tracing # echo 'foo == "Hello __rel_loc"' >> events/sample-trace/foo_rel_loc/filter
/sys/kernel/tracing # echo > trace
/sys/kernel/tracing # echo 1 >  events/sample-trace/foo_rel_loc/enable 
/sys/kernel/tracing # cat trace
# tracer: nop
#
# entries-in-buffer/entries-written: 4/4   #P:8
#
#                                _-----=> irqs-off
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
    event-sample-143     [001] .....   780.160062: foo_rel_loc: foo_rel_loc Hello __rel_loc, 752, 00000000,deadbeef
    event-sample-143     [001] .....   781.183814: foo_rel_loc: foo_rel_loc Hello __rel_loc, 753, 00000000,deadbeef
    event-sample-143     [001] .....   782.208076: foo_rel_loc: foo_rel_loc Hello __rel_loc, 754, 00000000,deadbeef
    event-sample-143     [001] .....   783.232116: foo_rel_loc: foo_rel_loc Hello __rel_loc, 755, 00000000,deadbeef

Thank you,

---

Masami Hiramatsu (5):
      tracing: Support __rel_loc relative dynamic data location attribute
      tracing: Add '__rel_loc' using trace event macros
      samples/trace_event: Add '__rel_loc' using sample event
      libtraceevent: Add __rel_loc relative location attribute support
      tools/perf: Add '__rel_loc' event field parsing support


 include/linux/trace_events.h                       |    1 
 include/trace/bpf_probe.h                          |   16 +++
 include/trace/perf.h                               |   16 +++
 include/trace/trace_events.h                       |  120 ++++++++++++++++++++
 kernel/trace/trace.h                               |    4 +
 kernel/trace/trace_events_filter.c                 |   32 +++++
 kernel/trace/trace_events_hist.c                   |   21 +++-
 kernel/trace/trace_events_inject.c                 |   11 ++
 samples/trace_events/trace-events-sample.c         |    3 +
 samples/trace_events/trace-events-sample.h         |   33 ++++++
 tools/lib/traceevent/event-parse.c                 |   59 +++++++---
 tools/lib/traceevent/event-parse.h                 |    5 +
 tools/lib/traceevent/parse-filter.c                |    5 +
 tools/perf/builtin-trace.c                         |    2 
 tools/perf/util/data-convert-bt.c                  |    2 
 tools/perf/util/evsel.c                            |    2 
 tools/perf/util/python.c                           |    2 
 .../perf/util/scripting-engines/trace-event-perl.c |    2 
 .../util/scripting-engines/trace-event-python.c    |    2 
 tools/perf/util/sort.c                             |    2 
 20 files changed, 310 insertions(+), 30 deletions(-)

--
Masami Hiramatsu (Linaro) <mhiramat@xxxxxxxxxx>