Re: [PATCH] perf report: Fix arm64 gap between kernel start and module end

From: Arnaldo Carvalho de Melo
Date: Tue Feb 09 2021 - 07:50:02 EST

Next message: Jiri Olsa: "Re: [PATCH v2] perf script: Simplify bool conversion"
Previous message: Andy Shevchenko: "Re: [PATCH v5 2/7] gpio: ep93xx: Fix single irqchip with multi gpiochips"
In reply to: Arnaldo Carvalho de Melo: "Re: [PATCH] perf report: Fix arm64 gap between kernel start and module end"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Em Tue, Feb 09, 2021 at 09:40:09AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Mar 30, 2020 at 10:18:10AM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Mon, Mar 30, 2020 at 10:11:29AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Mon, Mar 30, 2020 at 03:41:11PM +0800, Kemeng Shi escreveu:
> > > > diff --git a/tools/perf/arch/arm64/util/Build b/tools/perf/arch/arm64/util/Build
> > > > index 393b9895c..37cbfa5e9 100644
> > > > --- a/tools/perf/arch/arm64/util/Build
> > > > +++ b/tools/perf/arch/arm64/util/Build
> > > > @@ -2,6 +2,7 @@ libperf-y += header.o
> > > > libperf-y += tsc.o
> > > > libperf-y += sym-handling.o
> > > > libperf-y += kvm-stat.o
> > > > +libperf-y += machine.o
> > >
> > > You made the patch against an old perf codebase, right? This libperf-y
> > > above was changed to perf-y here:
> > >
> > > commit 5ff328836dfde0cef9f28c8b8791a90a36d7a183
> > > Author: Jiri Olsa <jolsa@xxxxxxxxxx>
> > > Date: Wed Feb 13 13:32:39 2019 +0100
> > >
> > > perf tools: Rename build libperf to perf
> > >
> > > ----
> > >
> > > I'm fixing this up, please check my perf/core branch later to see that
> > > all is working as intended.
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/core
> >
> > Here it is:
> >
> >
> > From 829b915d7d7eeafbe4af76dce19ccbdc743a43c8 Mon Sep 17 00:00:00 2001
> > From: Kemeng Shi <shikemeng@xxxxxxxxxx>
> > Date: Mon, 30 Mar 2020 15:41:11 +0800
> > Subject: [PATCH 1/1] perf symbols: Fix arm64 gap between kernel start and
> > module end
> >
> > During execution of command 'perf report' in my arm64 virtual machine,
> > this error message is showed:
> >
> > failed to process sample
> >
> > __symbol__inc_addr_samples(860): ENOMEM! sym->name=__this_module,
> > start=0x1477100, addr=0x147dbd8, end=0x80002000, func: 0
> >
> > The error is caused with path:
> > cmd_report
> > __cmd_report
> > perf_session__process_events
> > __perf_session__process_events
> > ordered_events__flush
> > __ordered_events__flush
> > oe->deliver (ordered_events__deliver_event)
> > perf_session__deliver_event
> > machines__deliver_event
> > perf_evlist__deliver_sample
> > tool->sample (process_sample_event)
> > hist_entry_iter__add
> > iter->add_entry_cb(hist_iter__report_callback)
> > hist_entry__inc_addr_samples
> > symbol__inc_addr_samples
> > __symbol__inc_addr_samples
> > h = annotated_source__histogram(src, evidx) (NULL)
> >
> > annotated_source__histogram failed is caused with path:
> > ...
> > hist_entry__inc_addr_samples
> > symbol__inc_addr_samples
> > symbol__hists
> > annotated_source__alloc_histograms
> > src->histograms = calloc(nr_hists, sizeof_sym_hist) (failed)
> >
> > Calloc failed as the symbol__size(sym) is too huge. As show in error
> > message: start=0x1477100, end=0x80002000, size of symbol is about 2G.
> >
> > This is the same problem as 'perf annotate: Fix s390 gap between kernel
> > end and module start (b9c0a64901d5bd)'. Perf gets symbol information from
> > /proc/kallsyms in __dso__load_kallsyms. A part of symbol in /proc/kallsyms
> > from my virtual machine is as follows:
> > #cat /proc/kallsyms | sort
> > ...
> > ffff000001475080 d rpfilter_mt_reg [ip6t_rpfilter]
> > ffff000001475100 d $d [ip6t_rpfilter]
> > ffff000001475100 d __this_module [ip6t_rpfilter]
> > ffff000080080000 t _head
> > ffff000080080000 T _text
> > ffff000080080040 t pe_header
> > ...
> >
> > Take line 'ffff000001475100 d __this_module [ip6t_rpfilter]' as example.
> > The start and end of symbol are both set to ffff000001475100 in
> > dso__load_all_kallsyms. Then symbols__fixup_end will set the end of symbol
> > to next big address to ffff000001475100 in /proc/kallsyms, ffff000080080000
> > in this example. Then sizeof of symbol will be about 2G and cause the
> > problem.
> >
> > The start of module in my machine is
> > ffff000000a62000 t $x [dm_mod]
> >
> > The start of kernel in my machine is
> > ffff000080080000 t _head
> >
> > There is a big gap between end of module and begin of kernel if a samll
> > amount of memory is used by module. And the last symbol in module will
> > have a large address range as caotaining the big gap.
> >
> > Give that the module and kernel text segment sequence may change in
> > the future, fix this by limiting range of last symbol in module and kernel
> > to 4K in arch arm64.
> >
> > Signed-off-by: Kemeng Shi <shikemeng@xxxxxxxxxx>
> > Cc: Alexander Shishkin <alexander.shishkin@xxxxxxxxxxxxxxx>
> > Cc: Hu Shiyuan <hushiyuan@xxxxxxxxxx>
> > Cc: Ian Rogers <irogers@xxxxxxxxxx>
> > Cc: Jiri Olsa <jolsa@xxxxxxxxxx>
> > Cc: Mark Rutland <mark.rutland@xxxxxxx>
> > Cc: Namhyung Kim <namhyung@xxxxxxxxxx>
> > Cc: Nick Desaulniers <ndesaulniers@xxxxxxxxxx>
> > Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> > Cc: Thomas Richter <tmricht@xxxxxxxxxxxxx>
> > Cc: hewenliang4@xxxxxxxxxx
> > Link: http://lore.kernel.org/lkml/33fd24c4-0d5a-9d93-9b62-dffa97c992ca@xxxxxxxxxx
> > [ refreshed the patch on current codebase ]
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx>
> > ---
> > tools/perf/arch/arm64/util/Build | 1 +
> > tools/perf/arch/arm64/util/machine.c | 26 ++++++++++++++++++++++++++
> > 2 files changed, 27 insertions(+)
> > create mode 100644 tools/perf/arch/arm64/util/machine.c
> >
> > diff --git a/tools/perf/arch/arm64/util/Build b/tools/perf/arch/arm64/util/Build
> > index 789956f76d85..5c13438c7bd4 100644
> > --- a/tools/perf/arch/arm64/util/Build
> > +++ b/tools/perf/arch/arm64/util/Build
> > @@ -1,4 +1,5 @@
> > perf-y += header.o
> > +perf-y += machine.o
> > perf-y += perf_regs.o
> > perf-$(CONFIG_DWARF) += dwarf-regs.o
> > perf-$(CONFIG_LOCAL_LIBUNWIND) += unwind-libunwind.o
> > diff --git a/tools/perf/arch/arm64/util/machine.c b/tools/perf/arch/arm64/util/machine.c
> > new file mode 100644
> > index 000000000000..94745131e89a
> > --- /dev/null
> > +++ b/tools/perf/arch/arm64/util/machine.c
> > @@ -0,0 +1,26 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include <stdio.h>
> > +#include "debug.h"
> > +#include "symbol.h"
> > +
> > +/* On arm64, kernel text segment start at high memory address,
> > + * for example 0xffff 0000 8xxx xxxx. Modules start at a low memory
> > + * address, like 0xffff 0000 00ax xxxx. When only samll amount of
> > + * memory is used by modules, gap between end of module's text segment
> > + * and start of kernel text segment may be reach 2G.
> > + * Therefore do not fill this gap and do not assign it to the kernel dso map.
> > + */
> > +
> > +#define SYMBOL_LIMIT (1 << 12) /* 4K */
> > +
> > +void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
> > +{
> > + if ((strchr(p->name, '[') && strchr(c->name, '[') == NULL) ||
> > + (strchr(p->name, '[') == NULL && strchr(c->name, '[')))
> > + /* Limit range of last symbol in module and kernel */
> > + p->end += SYMBOL_LIMIT;
> > + else
> > + p->end = c->start;
> > + pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
> > +}
>
> This breaks the build on powerpc 32-bit, I'm fixing it using the proper "%#" PRIx64 " style.
>
> - Arnaldo

Sorry, the build broke on powerpc 32-bit, but the above for arm also has
the same problem, I'll fix it there as well even it not breaking the
build as that is 64-bit specific (ARM case).

- Arnaldo

>
> 72 13.69 ubuntu:18.04-x-powerpc : FAIL powerpc-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
> arch/powerpc/util/machine.c: In function 'arch__symbols__fixup_end':
> arch/powerpc/util/machine.c:23:12: error: format '%lx' expects argument of type 'long unsigned int', but argument 6 has type 'u64 {aka long long unsigned int}' [-Werror=format=]
> pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
> ^
> /git/linux/tools/perf/util/debug.h:18:21: note: in definition of macro 'pr_fmt'
> #define pr_fmt(fmt) fmt
> ^~~
> /git/linux/tools/perf/util/debug.h:33:29: note: in expansion of macro 'pr_debugN'
> #define pr_debug4(fmt, ...) pr_debugN(4, pr_fmt(fmt), ##__VA_ARGS__)
> ^~~~~~~~~
> /git/linux/tools/perf/util/debug.h:33:42: note: in expansion of macro 'pr_fmt'
> #define pr_debug4(fmt, ...) pr_debugN(4, pr_fmt(fmt), ##__VA_ARGS__)
> ^~~~~~
> arch/powerpc/util/machine.c:23:2: note: in expansion of macro 'pr_debug4'
> pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
> ^~~~~~~~~
> cc1: all warnings being treated as errors
> /git/linux/tools/build/Makefile.build:139: recipe for target 'util' failed
> make[5]: *** [util] Error 2
> /git/linux/tools/build/Makefile.build:139: recipe for target 'powerpc' failed
> make[4]: *** [powerpc] Error 2
> /git/linux/tools/build/Makefile.build:139: recipe for target 'arch' failed
> make[3]: *** [arch] Error 2
> 73 30.47 ubuntu:18.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
>

--

- Arnaldo

Next message: Jiri Olsa: "Re: [PATCH v2] perf script: Simplify bool conversion"
Previous message: Andy Shevchenko: "Re: [PATCH v5 2/7] gpio: ep93xx: Fix single irqchip with multi gpiochips"
In reply to: Arnaldo Carvalho de Melo: "Re: [PATCH] perf report: Fix arm64 gap between kernel start and module end"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]