RE: [PATCH] perf evlist: fix memory corruption for Kernel PMU event

From: Song Bao Hua (Barry Song)
Date: Tue Oct 06 2020 - 02:39:49 EST




> -----Original Message-----
> From: Namhyung Kim [mailto:namhyung@xxxxxxxxxx]
> Sent: Tuesday, October 6, 2020 2:26 PM
> To: Song Bao Hua (Barry Song) <song.bao.hua@xxxxxxxxxxxxx>
> Cc: Andi Kleen <ak@xxxxxxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; Linuxarm
> <linuxarm@xxxxxxxxxx>; Peter Zijlstra <peterz@xxxxxxxxxxxxx>; Ingo Molnar
> <mingo@xxxxxxxxxx>; Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>; Mark
> Rutland <mark.rutland@xxxxxxx>; Alexander Shishkin
> <alexander.shishkin@xxxxxxxxxxxxxxx>; Jiri Olsa <jolsa@xxxxxxxxxx>; Adrian
> Hunter <adrian.hunter@xxxxxxxxx>; Alexey Budankov
> <alexey.budankov@xxxxxxxxxxxxxxx>
> Subject: Re: [PATCH] perf evlist: fix memory corruption for Kernel PMU event
>
> Hello,
>
> On Fri, Oct 2, 2020 at 12:02 PM Song Bao Hua (Barry Song)
> <song.bao.hua@xxxxxxxxxxxxx> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Andi Kleen [mailto:ak@xxxxxxxxxxxxxxx]
> > > Sent: Friday, October 2, 2020 12:07 PM
> > > To: Song Bao Hua (Barry Song) <song.bao.hua@xxxxxxxxxxxxx>
> > > Cc: linux-kernel@xxxxxxxxxxxxxxx; Linuxarm <linuxarm@xxxxxxxxxx>; Peter
> > > Zijlstra <peterz@xxxxxxxxxxxxx>; Ingo Molnar <mingo@xxxxxxxxxx>;
> Arnaldo
> > > Carvalho de Melo <acme@xxxxxxxxxx>; Mark Rutland
> > > <mark.rutland@xxxxxxx>; Alexander Shishkin
> > > <alexander.shishkin@xxxxxxxxxxxxxxx>; Jiri Olsa <jolsa@xxxxxxxxxx>;
> > > Namhyung Kim <namhyung@xxxxxxxxxx>; Adrian Hunter
> > > <adrian.hunter@xxxxxxxxx>; Alexey Budankov
> > > <alexey.budankov@xxxxxxxxxxxxxxx>
> > > Subject: Re: [PATCH] perf evlist: fix memory corruption for Kernel PMU
> event
> > >
> > > On Fri, Oct 02, 2020 at 12:57:29AM +1300, Barry Song wrote:
> > > > Commit 7736627b865d ("perf stat: Use affinity for closing file
> > > > descriptors") will use FD(evsel, cpu, thread) to read and write file
> > > > descriptors xyarray. For a kernel PMU event, this leads to serious
> > > > memory corruption and perf crash.
> > > > I have seen evlist->core.cpus->nr is 1 while evsel has cpus->nr with
> > > > the total number of CPUs. so xyarray which is allocated by
> > > > evlist->core.cpus->nr will get overflow. This leads to various
> > > > segmentation faults in perf tool for kernel PMU events, eg:
> > > > ./perf stat -e bus_cycles sleep 1
> > > > *** Error in `./perf': free(): invalid next size (fast):
> > > > 0x00000000401e6370 *** Aborted (core dumped)
> > >
> > > Thanks.
> > >
> > > I believe there is already a patch queued for this.
> >
> > Andi, thanks! Could you share the link or the commit ID? I'd like to take a
> look at the fix.
> > I could still reproduce this issue in the latest linus' tree and I didn't find any
> commit
> > related to this issue in linux-next and tip/perf/core.
>
> I think Andi was referring to this discussion which is not merged yet:
>
> https://lore.kernel.org/lkml/20200922031346.15051-2-liwei391@xxxxxxxxx
> m/
>
> I suggested a patch at the end. Can you please try it?

I tried the patch you suggested.

diff --git a/tools/lib/perf/evlist.c b/tools/lib/perf/evlist.c
index 2208444ecb44..cfcdbd7be066 100644
--- a/tools/lib/perf/evlist.c
+++ b/tools/lib/perf/evlist.c
@@ -45,6 +45,9 @@ static void __perf_evlist__propagate_maps(struct perf_evlist *evlist,
if (!evsel->own_cpus || evlist->has_user_cpus) {
perf_cpu_map__put(evsel->cpus);
evsel->cpus = perf_cpu_map__get(evlist->cpus);
+ } else if (!evsel->system_wide && perf_cpu_map__empty(evlist->cpus)) {
+ perf_cpu_map__put(evsel->cpus);
+ evsel->cpus = perf_cpu_map__get(evlist->cpus);
} else if (evsel->cpus != evsel->own_cpus) {
perf_cpu_map__put(evsel->cpus);
evsel->cpus = perf_cpu_map__get(evsel->own_cpus);

it did fix the crash I have seen on arm64. I'd prefer you put the below fixes tag in the commit log.
Fixes: 7736627b865d ("perf stat: Use affinity for closing file descriptors")
Perf stat began to crash from v5.4 kernel, so the fix should be backported to stable trees.

Thanks
Barry