Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

[Sedat Dilek]

On Sat, Feb 6, 2021 at 9:32 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
>
> On Sat, Feb 6, 2021 at 9:27 AM Yonghong Song <yhs@xxxxxx> wrote:
> >
> >
> >
> > On 2/5/21 10:52 PM, Sedat Dilek wrote:
> > > On Sat, Feb 6, 2021 at 7:26 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > >>
> > >> On Sat, Feb 6, 2021 at 6:53 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > >>>
> > >>> On Sat, Feb 6, 2021 at 6:44 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > >>>>
> > >>>> On Sat, Feb 6, 2021 at 4:34 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > >>>>>
> > >>>>> On Fri, Feb 5, 2021 at 10:54 PM Yonghong Song <yhs@xxxxxx> wrote:
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On 2/5/21 12:31 PM, Sedat Dilek wrote:
> > >>>>>>> On Fri, Feb 5, 2021 at 9:03 PM Yonghong Song <yhs@xxxxxx> wrote:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On 2/5/21 11:24 AM, Arnaldo Carvalho de Melo wrote:
> > >>>>>>>>> Em Fri, Feb 05, 2021 at 11:10:08AM -0800, Yonghong Song escreveu:
> > >>>>>>>>>> On 2/5/21 11:06 AM, Sedat Dilek wrote:
> > >>>>>>>>>>> On Fri, Feb 5, 2021 at 7:53 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > >>>>>>>>>>> Grepping through linux.git/tools I guess some BTF tools/libs need to
> > >>>>>>>>>>> know what BTF_INT_UNSIGNED is?
> > >>>>>>>>>
> > >>>>>>>>>> BTF_INT_UNSIGNED needs kernel support. Maybe to teach pahole to
> > >>>>>>>>>> ignore this for now until kernel infrastructure is ready.
> > >>>>>>>>>
> > >>>>>>>>> Yeah, I thought about doing that.
> > >>>>>>>>>
> > >>>>>>>>>> Not sure whether this information will be useful or not
> > >>>>>>>>>> for BTF. This needs to be discussed separately.
> > >>>>>>>>>
> > >>>>>>>>> Maybe search for the rationale for its introduction in DWARF.
> > >>>>>>>>
> > >>>>>>>> In LLVM, we have:
> > >>>>>>>>      uint8_t BTFEncoding;
> > >>>>>>>>      switch (Encoding) {
> > >>>>>>>>      case dwarf::DW_ATE_boolean:
> > >>>>>>>>        BTFEncoding = BTF::INT_BOOL;
> > >>>>>>>>        break;
> > >>>>>>>>      case dwarf::DW_ATE_signed:
> > >>>>>>>>      case dwarf::DW_ATE_signed_char:
> > >>>>>>>>        BTFEncoding = BTF::INT_SIGNED;
> > >>>>>>>>        break;
> > >>>>>>>>      case dwarf::DW_ATE_unsigned:
> > >>>>>>>>      case dwarf::DW_ATE_unsigned_char:
> > >>>>>>>>        BTFEncoding = 0;
> > >>>>>>>>        break;
> > >>>>>>>>
> > >>>>>>>> I think DW_ATE_unsigned can be ignored in pahole since
> > >>>>>>>> the default encoding = 0. A simple comment is enough.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> Yonghong Son, do you have a patch/diff for me?
> > >>>>>>
> > >>>>>> Looking at error message from log:
> > >>>>>>
> > >>>>>>    LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > >>>>>> .tmp_vmlinux.btf
> > >>>>>> [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > >>>>>> Encountered error while encoding BTF.
> > >>>>>>
> > >>>>>> Not exactly what is the root cause. Maybe bt->bit_size is not
> > >>>>>> encoded correctly. Could you put vmlinux (in the above it is
> > >>>>>> .tmp_vmlinux.btf) somewhere, I or somebody else can investigate
> > >>>>>> and provide a proper fix.
> > >>>>>>
> > >>>>>
> > >>>>> [ TO: Masahiro ]
> > >>>>>
> > >>>>> Thanks for taking care Yonghong - hope this is your first name, if not
> > >>>>> I am sorry.
> > >>>>> In case of mixing my first and last name you will make me female -
> > >>>>> Dilek is a Turkish female first name :-).
> > >>>>> So, in some cultures you need to be careful.
> > >>>>>
> > >>>>> Anyway... back to business and facts.
> > >>>>>
> > >>>>> Out of frustration I killed my last build via `make distclean`.
> > >>>>> The whole day I tested diverse combination of GCC-10 and LLVM-12
> > >>>>> together with BTF Kconfigs, selfmade pahole, etc.
> > >>>>>
> > >>>>> I will do ne run with some little changes:
> > >>>>>
> > >>>>> #1: Pass LLVM_IAS=1 to make (means use Clang's Integrated ASsembler -
> > >>>>> as per Nick this leads to the same error - should be unrelated)
> > >>>>> #2: I did: DEBUG_INFO_COMPRESSED y -> n
> > >>>>>
> > >>>>> #2 I did in case you need vmlinux and I have to upload - I will
> > >>>>> compress the resulting vmlinux with ZSTD.
> > >>>>> You need vmlinux or .tmp_vmlinux.btf file?
> > >>>>> Nick was not allowed from his company to download from a Dropbox link.
> > >>>>> So, as an alternative I can offer GoogleDrive...
> > >>>>> ...or bomb into your INBOX :-).
> > >>>>>
> > >>>>> Now, why I CCed Masahiro:
> > >>>>>
> > >>>>> In case of ERRORs when running `scripts/link-vmlinux.sh` above files
> > >>>>> will be removed.
> > >>>>>
> > >>>>> Last, I found a hack to bypass this - means to keep these files (I
> > >>>>> need to check old emails).
> > >>>>>
> > >>>>> Masahiro, you see a possibility to have a way to keep these files in
> > >>>>> case of ERRORs without doing hackery?
> > >>>>>
> > >>>>>  From a previous post in this thread:
> > >>>>>
> > >>>>> + info BTF .btf.vmlinux.bin.o
> > >>>>> + [  != silent_ ]
> > >>>>> + printf   %-7s %s\n BTF .btf.vmlinux.bin.o
> > >>>>>   BTF     .btf.vmlinux.bin.o
> > >>>>> + LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
> > >>>>> [2] INT long unsigned int Error emitting BTF type
> > >>>>> Encountered error while encoding BTF.
> > >>>>> + llvm-objcopy --only-section=.BTF --set-section-flags
> > >>>>> .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
> > >>>>> ...
> > >>>>> + info BTFIDS vmlinux
> > >>>>> + [  != silent_ ]
> > >>>>> + printf   %-7s %s\n BTFIDS vmlinux
> > >>>>>   BTFIDS  vmlinux
> > >>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > >>>>> FAILED: load BTF from vmlinux: Invalid argument
> > >>>>> + on_exit
> > >>>>> + [ 255 -ne 0 ]
> > >>>>> + cleanup
> > >>>>> + rm -f .btf.vmlinux.bin.o
> > >>>>> + rm -f .tmp_System.map
> > >>>>> + rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
> > >>>>> .tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
> > >>>>> .tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
> > >>>>> 2.o
> > >>>>> + rm -f System.map
> > >>>>> + rm -f vmlinux
> > >>>>> + rm -f vmlinux.o
> > >>>>> make[3]: *** [Makefile:1166: vmlinux] Error 255
> > >>>>>
> > >>>>> ^^^ Look here.
> > >>>>>
> > >>>>
> > >>>> With this diff:
> > >>>>
> > >>>> $ git diff scripts/link-vmlinux.sh
> > >>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> > >>>> index eef40fa9485d..40f1b6aae553 100755
> > >>>> --- a/scripts/link-vmlinux.sh
> > >>>> +++ b/scripts/link-vmlinux.sh
> > >>>> @@ -330,7 +330,7 @@ vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
> > >>>> # fill in BTF IDs
> > >>>> if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
> > >>>>         info BTFIDS vmlinux
> > >>>> -       ${RESOLVE_BTFIDS} vmlinux
> > >>>> +       ##${RESOLVE_BTFIDS} vmlinux
> > >>>> fi
> > >>>>
> > >>>> if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
> > >>>>
> > >>>> This files are kept - not removed:
> > >>>>
> > >>>> $ LC_ALL=C ll .*btf* vmlinux vmlinux.o
> > >>>> -rwxr-xr-x 1 dileks dileks  31M Feb  6 06:37 .btf.vmlinux.bin.o
> > >>>> -rwxr-xr-x 1 dileks dileks 348M Feb  6 06:37 .tmp_vmlinux.btf
> > >>>> -rwxr-xr-x 1 dileks dileks 348M Feb  6 06:37 vmlinux
> > >>>> -rw-r--r-- 1 dileks dileks 344M Feb  6 06:37 vmlinux.o
> > >>>>
> > >>>> Pleas let me know where to upload - Dropbox or GoogleDrive or
> > >>>> elsewhere and give me a link.
> > >>>>
> > >>>
> > >>>
> > >>> WOW, ZSTD is great :-).
> > >>>
> > >>> $ zstd -19 -T0 -v vmlinux
> > >>> *** zstd command line interface 64-bits v1.4.8, by Yann Collet ***
> > >>> Note: 2 physical core(s) detected
> > >>> vmlinux              : 22.71%   (364466016 => 82784801 bytes, vmlinux.zst)
> > >>>
> > >>> $ du -m vmlinux*
> > >>> 348     vmlinux
> > >>> 79      vmlinux.zst
> > >>>
> > >>
> > >> Dropbox link:
> > >> https://www.dropbox.com/sh/kvyh8ps7na0r1h5/AABfyNfDZ2bESse_bo4h05fFa?dl=0
> > >>
> > >> I hope this is public available.
> > >>
> > >
> > > Inspecting vmlinux with llvm-dwarf:
> > >
> > > $ /opt/llvm-toolchain/bin/llvm-dwarfdump vmlinux | grep DW_AT_name |
> > > grep DW_ATE_ | sort | uniq
> > >                 DW_AT_name      ("DW_ATE_signed_1")
> > >                 DW_AT_name      ("DW_ATE_signed_16")
> > >                 DW_AT_name      ("DW_ATE_signed_32")
> > >                 DW_AT_name      ("DW_ATE_signed_64")
> > >                 DW_AT_name      ("DW_ATE_signed_8")
> > >                 DW_AT_name      ("DW_ATE_unsigned_1")
> > >                 DW_AT_name      ("DW_ATE_unsigned_128")
> > >                 DW_AT_name      ("DW_ATE_unsigned_16")
> > >                 DW_AT_name      ("DW_ATE_unsigned_24")
> > >                 DW_AT_name      ("DW_ATE_unsigned_32")
> > >                 DW_AT_name      ("DW_ATE_unsigned_40")
> > >                 DW_AT_name      ("DW_ATE_unsigned_64")
> > >                 DW_AT_name      ("DW_ATE_unsigned_8")
> > >
> > > - Sedat -
> >
> > Thanks for the above dropbot link, I am able to reproduce the issue.
> >
> > I tried to use latest llvm + Nick's patch + latest pahole + dwarf5
> > config option to compile kernel with LLVM=1 LLVM_IAS=1, somehow
> > I did not hit the issue. It complained like
> >
> >    MODPOST vmlinux.symvers
> > WARNING: modpost: vmlinux.o(.text+0x6ce73): Section mismatch in
> > reference from the function __nodes
> > _weight() to the variable .init.data:numa_nodes_parsed
> > The function __nodes_weight() references
> > the variable __initdata numa_nodes_parsed.
> > This is often because __nodes_weight lacks a __initdata
> > annotation or the annotation of numa_nodes_parsed is wrong.
> >
> > but otherwise compilation is fine.
> >
> > With the above vmlinux, the issue appears to be handling
> > DW_ATE_signed_1, DW_ATE_unsigned_{1,24,40}.
> >
> > The following patch should fix the issue:
> >
> > -bash-4.4$ git diff
> >
> > diff --git a/dwarf_loader.c b/dwarf_loader.c
> >
> > index b73d786..0341b5e 100644
> >
> > --- a/dwarf_loader.c
> >
> > +++ b/dwarf_loader.c
> >
> > @@ -467,8 +467,16 @@ static struct base_type *base_type__new(Dwarf_Die
> > *die, struct cu *cu)
> >
> >
> >          if (bt != NULL) {
> >
> >                  tag__init(&bt->tag, cu, die);
> >
> > -               bt->name = strings__add(strings, attr_string(die,
> > DW_AT_name));
> > -               bt->bit_size = attr_numeric(die, DW_AT_byte_size) * 8;
> >
> > +               const char *name = attr_string(die, DW_AT_name);
> >
> > +               bt->name = strings__add(strings, name);
> >
> > +               /* DW_ATE_unsigned_1 has DW_AT_byte_size == 0.
> >
> > +                * specially process it.
> >
> > +                */
> >
> > +               if (strcmp(name, "DW_ATE_unsigned_1") == 0)
> >
> > +                       bt->bit_size = 1;
> >
> > +               else
> >
> > +                       bt->bit_size = attr_numeric(die,
> > DW_AT_byte_size) * 8;
> > +
> >
> >                  uint64_t encoding = attr_numeric(die, DW_AT_encoding);
> >                  bt->is_bool = encoding == DW_ATE_boolean;
> >                  bt->is_signed = encoding == DW_ATE_signed;
> > diff --git a/libbtf.c b/libbtf.c
> > index 9f76283..b5aa077 100644
> > --- a/libbtf.c
> > +++ b/libbtf.c
> > @@ -367,13 +367,32 @@ static void btf_log_func_param(const struct
> > btf_elf *btfe,
> >          }
> >   }
> >
> > +/* btf requires power-of-2 bytes, yet dwarf may have something like
> > + * DW_ATE_unsigned_24 which encodes as 24 bits (3 bytes).
> > + */
> > +static int bits_to_int_bytes(uint16_t bit_size)
> > +{
> > +       if (bit_size <= 8)
> > +               return 1;
> > +       if (bit_size <= 16)
> > +               return 2;
> > +       if (bit_size <= 32)
> > +               return 4;
> > +       if (bit_size <= 64)
> >
> > +               return 8;
> >
> > +       if (bit_size <= 128)
> > +               return 16;
> > +       /* BTF supports upto 16byte int (__int128). */
> > +       return -1;
> > +}
> > +
> >   int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct
> > base_type *bt,
> >                                 const char *name)
> >   {
> >          struct btf *btf = btfe->btf;
> >          const struct btf_type *t;
> >          uint8_t encoding = 0;
> > -       int32_t id;
> > +       int32_t id, nbytes;
> >
> >          if (bt->is_signed) {
> >                  encoding = BTF_INT_SIGNED;
> > @@ -384,7 +403,13 @@ int32_t btf_elf__add_base_type(struct btf_elf
> > *btfe, const struct base_type *b
> > t,
> >                  return -1;
> >          }
> > -       id = btf__add_int(btf, name, BITS_ROUNDUP_BYTES(bt->bit_size),
> > encoding);
> > +       nbytes = bits_to_int_bytes(bt->bit_size);
> > +       if (nbytes < 0) {
> > +               fprintf(stderr, "not supported bit_size %hu\n",
> > bt->bit_size);
> > +               return -1;
> > +       }
> > +
> > +       id = btf__add_int(btf, name, nbytes, encoding);
> >          if (id < 0) {
> >                  btf_elf__log_err(btfe, BTF_KIND_INT, name, true, "Error
> > emitting BTF type");
> >          } else {
> > -bash-4.4$
> >
> > Please help do a test. I can submit a formal patch tomorrow.
>
> Thanks for the patch.
>
> Can you attach the diff as Gmail has totally truncated/malformed it?
>
> For the Linux breakage - you will need some additional Clang specific patches.
> Is this Linux 5.11-rcX?
> The "Blocking bugs" are listed in the first post of "Linux 5.11 release cycle".
> Hope this helps.
>
> This is cool co-working :-).
>
> - Sedat -
>
> [1] https://github.com/ClangBuiltLinux/linux/issues/1228

With the attached diff and new selfmade pahole looks good here.

Passed (see line-numbers):

11090:+ info LD .tmp_vmlinux.btf
11099:+ info BTF .btf.vmlinux.bin.o
11103:+ LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
11121:+ info LD .tmp_vmlinux.kallsyms1
11139:+ info KSYMS .tmp_vmlinux.kallsyms1.S
11145:+ info AS .tmp_vmlinux.kallsyms1.S
11160:+ info LD .tmp_vmlinux.kallsyms2
11178:+ info KSYMS .tmp_vmlinux.kallsyms2.S
11184:+ info AS .tmp_vmlinux.kallsyms2.S
11200:+ info LD vmlinux
11210:+ info BTFIDS vmlinux
11216:+ info SORTTAB vmlinux

Still building linux-kernel...

Will report later if I was able to boot on bare metal.

- Sedat -
diff --git a/dwarf_loader.c b/dwarf_loader.c
index b73d7867e1e6..aba14679a07e 100644
--- a/dwarf_loader.c
+++ b/dwarf_loader.c
@@ -467,8 +467,15 @@ static struct base_type *base_type__new(Dwarf_Die *die, struct cu *cu)
 
 	if (bt != NULL) {
 		tag__init(&bt->tag, cu, die);
-		bt->name = strings__add(strings, attr_string(die, DW_AT_name));
-		bt->bit_size = attr_numeric(die, DW_AT_byte_size) * 8;
+		const char *name = attr_string(die, DW_AT_name);
+		bt->name = strings__add(strings, name);
+		/* DW_ATE_unsigned_1 has DW_AT_byte_size == 0.
+		* specially process it.
+		*/
+		if (strcmp(name, "DW_ATE_unsigned_1") == 0)
+			bt->bit_size = 1;
+		else
+			bt->bit_size = attr_numeric(die, DW_AT_byte_size) * 8;
 		uint64_t encoding = attr_numeric(die, DW_AT_encoding);
 		bt->is_bool = encoding == DW_ATE_boolean;
 		bt->is_signed = encoding == DW_ATE_signed;
diff --git a/libbtf.c b/libbtf.c
index 9f7628304495..44db29e83433 100644
--- a/libbtf.c
+++ b/libbtf.c
@@ -367,13 +367,32 @@ static void btf_log_func_param(const struct btf_elf *btfe,
 	}
 }
 
+/* btf requires power-of-2 bytes, yet dwarf may have something like
+ * DW_ATE_unsigned_24 which encodes as 24 bits (3 bytes).
+ */
+static int bits_to_int_bytes(uint16_t bit_size)
+{
+       if (bit_size <= 8)
+               return 1;
+       if (bit_size <= 16)
+               return 2;
+       if (bit_size <= 32)
+               return 4;
+       if (bit_size <= 64)
+               return 8;
+       if (bit_size <= 128)
+               return 16;
+       /* BTF supports upto 16byte int (__int128). */
+       return -1;
+}
+
 int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct base_type *bt,
 			       const char *name)
 {
 	struct btf *btf = btfe->btf;
 	const struct btf_type *t;
 	uint8_t encoding = 0;
-	int32_t id;
+	int32_t id, nbytes;
 
 	if (bt->is_signed) {
 		encoding = BTF_INT_SIGNED;
@@ -384,7 +403,13 @@ int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct base_type *bt,
 		return -1;
 	}
 
-	id = btf__add_int(btf, name, BITS_ROUNDUP_BYTES(bt->bit_size), encoding);
+	nbytes = bits_to_int_bytes(bt->bit_size);
+	if (nbytes < 0) {
+		fprintf(stderr, "not supported bit_size %hu\n", bt->bit_size);
+		return -1;
+	}
+
+	id = btf__add_int(btf, name, nbytes, encoding);
 	if (id < 0) {
 		btf_elf__log_err(btfe, BTF_KIND_INT, name, true, "Error emitting BTF type");
 	} else {