Re: [PATCH v2] kallsyms: strip LTO-only suffixes from promoted global functions

[Yonghong Song]

On 7/7/23 5:23 PM, Yonghong Song wrote:
> On 7/7/23 3:27 PM, Nick Desaulniers wrote:
> > On Wed, Jun 28, 2023 at 11:19 AM Yonghong Song <yhs@xxxxxx> wrote:
> > > Commit 6eb4bd92c1ce ("kallsyms: strip LTO suffixes from static 
> > > functions")
> > > stripped all function/variable suffixes started with '.' regardless
> > > of whether those suffixes are generated at LTO mode or not. In fact,
> > > as far as I know, in LTO mode, when a static function/variable is
> > > promoted to the global scope, '.llvm.<...>' suffix is added.
> > >
> > > The existing mechanism breaks live patch for a LTO kernel even if
> > > no <symbol>.llvm.<...> symbols are involved. For example, for the 
> > > following
> > > kernel symbols:
> > >    $ grep bpf_verifier_vlog /proc/kallsyms
> > >    ffffffff81549f60 t bpf_verifier_vlog
> > >    ffffffff8268b430 d bpf_verifier_vlog._entry
> > >    ffffffff8282a958 d bpf_verifier_vlog._entry_ptr
> > >    ffffffff82e12a1f d bpf_verifier_vlog.__already_done
> > > 'bpf_verifier_vlog' is a static function. '_entry', '_entry_ptr' and
> > > '__already_done' are static variables used inside 'bpf_verifier_vlog',
> > > so llvm promotes them to file-level static with prefix 
> > > 'bpf_verifier_vlog.'.
> > > Note that the func-level to file-level static function promotion also
> > > happens without LTO.
> > >
> > > Given a symbol name 'bpf_verifier_vlog', with LTO kernel, current 
> > > mechanism will
> > > return 4 symbols to live patch subsystem which current live patching
> > > subsystem cannot handle it. With non-LTO kernel, only one symbol
> > > is returned.
> > >
> > > In [1], we have a lengthy discussion, the suggestion is to separate two
> > > cases:
> > >    (1). new symbols with suffix which are generated regardless of 
> > > whether
> > >         LTO is enabled or not, and
> > >    (2). new symbols with suffix generated only when LTO is enabled.
> > >
> > > The cleanup_symbol_name() should only remove suffixes for case (2).
> > > Case (1) should not be changed so it can work uniformly with or 
> > > without LTO.
> > >
> > > This patch removed LTO-only suffix '.llvm.<...>' so live patching and
> > > tracing should work the same way for non-LTO kernel.
> > > The cleanup_symbol_name() in scripts/kallsyms.c is also changed to 
> > > have the same
> > > filtering pattern so both kernel and kallsyms tool have the same
> > > expectation on the order of symbols.
> > >
> > >   [1] 
> > > https://lore.kernel.org/live-patching/20230615170048.2382735-1-song@xxxxxxxxxx/T/#u
> > >
> > > Fixes: 6eb4bd92c1ce ("kallsyms: strip LTO suffixes from static 
> > > functions")
> > > Reported-by: Song Liu <song@xxxxxxxxxx>
> > > Signed-off-by: Yonghong Song <yhs@xxxxxx>
> >
> > Thanks for the patch and improving live patch with LTO.  Looking back
> > at the internal report that resulted in
> > commit 6eb4bd92c1ce ("kallsyms: strip LTO suffixes from static 
> > functions")
> > your version was what I originally had.  I did not leave a comment as
> > to why I changed it when I sent it 2 years ago, and no longer recall
> > the reason.
> >
> >
> > Reviewed-by: Nick Desaulniers <ndesaulniers@xxxxxxxxxx>
>
> Nick, thanks for the review. I am not sure how this patch could
> be merged into the mainline. I checked your patch
>     6eb4bd92c1ce ("kallsyms: strip LTO suffixes from static functions")
> and it looks like the patch was merged by Kees Cook.
>
> I added Kees in the 'To' list. Kees, could you help merge
> this patch if you are okay with the change?

Looks like Kees might be on vacation. Nick, Petr, do you
have any suggestions who could take this patch? Thanks!

> Thanks!
>
> Yonghong
>
> > > ---
> > >   kernel/kallsyms.c  | 5 ++---
> > >   scripts/kallsyms.c | 6 +++---
> > >   2 files changed, 5 insertions(+), 6 deletions(-)
> > >
> > > Changelogs:
> > >    v1 -> v2:
> > >      . add 'Reported-by: Song Liu <song@xxxxxxxxxx>'
> > >      . also fix in scripts/kallsyms.c.
> > >
> > > diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
> > > index 77747391f49b..4874508bb950 100644
> > > --- a/kernel/kallsyms.c
> > > +++ b/kernel/kallsyms.c
> > > @@ -174,11 +174,10 @@ static bool cleanup_symbol_name(char *s)
> > >           * LLVM appends various suffixes for local functions and 
> > > variables that
> > >           * must be promoted to global scope as part of LTO.  This 
> > > can break
> > >           * hooking of static functions with kprobes. '.' is not a valid
> > > -        * character in an identifier in C. Suffixes observed:
> > > +        * character in an identifier in C. Suffixes only in LLVM LTO 
> > > observed:
> > >           * - foo.llvm.[0-9a-f]+
> > > -        * - foo.[0-9a-f]+
> > >           */
> > > -       res = strchr(s, '.');
> > > +       res = strstr(s, ".llvm.");
> > >          if (res) {
> > >                  *res = '\0';
> > >                  return true;
> > > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > > index 0d2db41177b2..13af6d0ff845 100644
> > > --- a/scripts/kallsyms.c
> > > +++ b/scripts/kallsyms.c
> > > @@ -346,10 +346,10 @@ static void cleanup_symbol_name(char *s)
> > >           * ASCII[_]   = 5f
> > >           * ASCII[a-z] = 61,7a
> > >           *
> > > -        * As above, replacing '.' with '\0' does not affect the main 
> > > sorting,
> > > -        * but it helps us with subsorting.
> > > +        * As above, replacing the first '.' in ".llvm." with '\0' 
> > > does not
> > > +        * affect the main sorting, but it helps us with subsorting.
> > >           */
> > > -       p = strchr(s, '.');
> > > +       p = strstr(s, ".llvm.");
> > >          if (p)
> > >                  *p = '\0';
> > >   }
> > > --
> > > 2.34.1