Re: Issue using faddr2line on kernel modules

[Kaiwan N Billimoria]

Hi Josh,

Actually your first patch - the one you mentioned had other issues -
worked perfectly when applied:

scripts/faddr2line ./oops_tryv2.ko do_the_work+0x16f/0x194
do_the_work+0x16f/0x0000000000000194:
do_the_work at <...>/oops_tryv2/oops_tryv2.c:62

The second one still failed in the same manner:

scripts/faddr2line ./oops_tryv2.ko do_the_work+0x16f/0x194
bad symbol size: base: 0x0000000000000000 end: 0x0000000000000000

So, is it possible to fixup issues with the first version?
What are these issues?

Thanks,
Kaiwan.



On Tue, Jan 18, 2022 at 1:57 AM Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:
>
> On Mon, Jan 17, 2022 at 11:48:39AM -0800, Josh Poimboeuf wrote:
> > On Mon, Jan 17, 2022 at 10:27:14AM +0530, Kaiwan N Billimoria wrote:
> > > Hi there,
> > >
> > > Am researching using the cool faddr2line script to help debug Oops'es
> > > from kernel modules..
> > > I find it works just fine when used against the unstripped vmlinux
> > > with debug symbols.
> > >
> > > My use case is for a kernel module which Oopses, though. Here's my scenario:
> > > I built a module on a custom debug kernel (5.10.60) with most debug
> > > options enabled...
> > > KASLR is enabled by default as well.
> > >
> > > A test kernel module Oopses on my x86_64 guest running this kernel with:
> > > RIP: 0010:do_the_work+0x15b/0x174 [oops_tryv2]
> > >
> > > So, i try this:
> > >
> > > $ <...>/linux-5.10.60/scripts/faddr2line ./oops_tryv2.ko do_the_work+0x15b/0x174
> > > bad symbol size: base: 0x0000000000000000 end: 0x0000000000000000
> > > $
> > >
> > > (It works fine with addr2line though!).
> > > Now I think I've traced the faddr2line script's failure to locate
> > > anything down to this:
> > > ...
> > > done < <(${NM} -n $objfile | awk -v fn=$func -v end=$file_end '$3 ==
> > > fn { found=1; line    =$0; start=$1; next } found == 1 { found=0;
> > > print line, "0x"$1 } END {if (found == 1) print     line, end; }')
> > >
> > > The nm output is:
> > > $ nm -n ./oops_tryv2.ko |grep -i do_the_work
> > > 0000000000000000 t do_the_work
> > > $
> > >
> > > nm shows the text addr as 0x0; this is obviously incorrect (same 0x0
> > > with objdump -d on the module).
> > > Am I missing something? Any suggestions as to what I can try, to get
> > > faddr2line working?
> >
> > Hi Kaiwan,
> >
> > Thanks for reporting this issue.  The module text address of 0x0 is not
> > necessarily incorrect, as the address is relative the the module, where
> > all text usually starts at zero.
> >
> > I was able to recreate this problem using a module which only has a
> > single function in .text.  Does this fix it?
>
> Actually, that patch has other problems.  Try this one?
>
> ----
>
> From: Josh Poimboeuf <jpoimboe@xxxxxxxxxx>
> Subject: [PATCH] scripts/faddr2line: Only look for text symbols when
>  calculating function size
>
> With the following commit:
>
>   efdb4167e676 ("scripts/faddr2line: Fix "size mismatch" error")
>
> ... it was discovered that faddr2line can't just read a function's ELF
> size, because that wouldn't match the kallsyms function size which is
> printed in the stack trace.  The kallsyms size includes any padding
> after the function, whereas the ELF size does not.
>
> So faddr2line has to manually calculate the size of a function similar
> to how kallsyms does.  It does so by starting with a sorted list of
> symbols and subtracting the function address from the subsequent
> symbol's address.
>
> That calculation is broken in the case where the function is the last
> (or only) symbol in the .text section.  The next symbol in the sorted
> list might actually be a data symbol, which can break the function size
> detection:
>
>   $ scripts/faddr2line sound/soundcore.ko sound_devnode+0x5/0x35
>   bad symbol size: base: 0x0000000000000000 end: 0x0000000000000000
>
> Similar breakage can occur when reading from a .o file.
>
> Fix it by only looking for text symbols.
>
> Fixes: efdb4167e676 ("scripts/faddr2line: Fix "size mismatch" error")
> Reported-by: Kaiwan N Billimoria <kaiwan.billimoria@xxxxxxxxx>
> Signed-off-by: Josh Poimboeuf <jpoimboe@xxxxxxxxxx>
> ---
>  scripts/faddr2line | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/faddr2line b/scripts/faddr2line
> index 6c6439f69a72..2a130134f1e6 100755
> --- a/scripts/faddr2line
> +++ b/scripts/faddr2line
> @@ -189,7 +189,7 @@ __faddr2line() {
>
>                 DONE=1
>
> -       done < <(${NM} -n $objfile | awk -v fn=$func -v end=$file_end '$3 == fn { found=1; line=$0; start=$1; next } found == 1 { found=0; print line, "0x"$1 } END {if (found == 1) print line, end; }')
> +       done < <(${NM} -n $objfile | awk -v fn=$func -v end=$file_end '$2 !~ /[Tt]/ {next} $3 == fn { found=1; line=$0; start=$1; next } found == 1 { found=0; print line, "0x"$1 } END {if (found == 1) print line, end; }')
>  }
>
>  [[ $# -lt 2 ]] && usage
> --
> 2.31.1
>