Re: [BUG] perf: No samples found when using kcore + coresight

[Yang Shi]

On Thu, Mar 9, 2023 at 3:39 AM Leo Yan <leo.yan@xxxxxxxxxx> wrote:
>
> Hi Yang,
>
> On Wed, Mar 08, 2023 at 11:56:38AM -0800, Yang Shi wrote:
>
> [...]
>
> > > Dumping raw events could show the events from the bad data file. But
> > > it has zero samples after event collapse.
> > >
> > > The only difference is --kcore inserted a new text_poke dummy event.
> > > It seems coresight also inserted a dummy event with my command but
> > > your command didn't. So it seems like the two dummy events confused
> > > event collapse.
> > >
> > > The text_poke dummy event is added by commit
> > > f42c0ce573df79d1b8bd169008c994dcdd43585a ("perf record: Always get
> > > text_poke events with --kcore option"). If I reverted this commit,
> > > then it works. But I'm not sure whether this is the right fix or real
> > > root cause or not. Or coresight shouldn't insert its own dummy event?
> >
> > It seems like coresight needs to insert the dummy event if
> > full_auxtrace is on IIUC. So it sounds like event collapse can't
> > handle such a case?
>
> I am struggling to understand the meaning "event collapse" :)

I mean report__collapse_hists(). Since dumping raw events is fine, so
it seems like report__collapse_hists() returns 0 samples after
collapse.

>
> I reviewed your shared dump, the bad and good perf data both contain the
> dummy event with 'text_poke = 1'.  Could you confirm the shared dump in
> your previous email is correct or not?

Oops, sorry. I pasted the wrong log. The good one looks like
(generated by v5.19):

# captured on    : Wed Mar  8 18:02:58 2023
# header version : 1
# data offset    : 408
# data size      : 22640
# feat offset    : 23048
# hostname : fedora
# os release : 6.2.0-coresight+
# perf version : 5.19.g3d7cb6b04c3f
# arch : aarch64
# nrcpus online : 128
# nrcpus avail : 128
# cpuid : 0x00000000c00fac30
# total memory : 2108862504 kB
# cmdline : /home/yshi/linux/tools/perf/perf record -e
cs_etm/@tmc_etf63/k --kcore --per-thread -- taskset --cpu-list 1 uname
# event : name = cs_etm/@tmc_etf63/k, , id = { 3832 }, type = 9, size
= 128, { sample_period, sample_freq } = 1, sample_type =
IP|TID|IDENTIFIER, read_format = ID, d
isabled = 1, exclude_user = 1, exclude_hv = 1, enable_on_exec = 1,
sample_id_all = 1, { bp_len, config2 } = 0x12792918
# event : name = dummy:u, , id = { 3833 }, type = 1, size = 128,
config = 0x9, { sample_period, sample_freq } = 1, sample_type =
IP|TID|IDENTIFIER, read_format = ID,
 disabled = 1, exclude_kernel = 1, exclude_hv = 1, mmap = 1, comm = 1,
enable_on_exec = 1, task = 1, sample_id_all = 1, exclude_guest = 1,
mmap2 = 1, comm_exec = 1,
context_switch = 1, ksymbol = 1, bpf_event = 1
# CPU_TOPOLOGY info available, use -I to display
# NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: armv8_pmuv3_0 = 8, software = 1, arm_cmn_0 = 10,
uprobe = 7, cs_etm = 9, breakpoint = 5, tracepoint = 2, arm_cmn_1 =
11, kprobe = 6
# contains AUX area data (e.g. instruction trace)
# CACHE info available, use -I to display
# time of first sample : 18446744073.709551
# time of last sample : 18446744073.709551
# sample duration :      0.000 ms
# MEM_TOPOLOGY info available, use -I to display
# missing features: TRACING_DATA CPUDESC BRANCH_STACK GROUP_DESC STAT
CLOCKID DIR_FORMAT COMPRESSED CPU_PMU_CAPS CLOCK_DATA HYBRID_TOPOLOGY
HYBRID_CPU_PMU_CAPS

>
> > I also tried v5.19 (before "perf record: Always
> > get text_poke events with --kcore option", which was merged in v6.0),
> > it works. So it seems like a regression.
>
> Yeah, we need to fix it.  I am not sure the Linux kernel for Arm64
> supports text poke or not (kernel needs some specific handling when
> alter instructions), the kernel change is the prerequisites.
>
> On the other hand, in the current code cs-etm misses to handle the
> event PERF_RECORD_TEXT_POKE in the function cs_etm__process_event().
> This might be the cause for the failure.
>
> Do you mind to share the bad perf.data file with James and me?

Please check the attachment out. Thanks for looking into this problem.

>
> Thanks,
> Leo

Attachment: perf.data.broken
Description: Binary data