Re: [RFC PATCH net-next] tools/bpf: fix build with binutils >= 2.28

[Quentin Monnet]

2017-12-20 18:32 UTC+0000 ~ Roman Gushchin <guro@xxxxxx>
> On Tue, Dec 19, 2017 at 04:22:51PM +0000, Quentin Monnet wrote:
>> 2017-12-19 16:10 UTC+0000 ~ Roman Gushchin <guro@xxxxxx>
>>> On Tue, Dec 19, 2017 at 03:57:02PM +0000, Quentin Monnet wrote:
>>>> Hi Roman, thanks for working on this!
>>>>
>>>>
>>>> I discussed this issue with Jakub recently, and one suggestion he had
>>>> was to look in tools/build/feature to add a new "feature", by trying to
>>>> compile short programs, for making the distinction between binutils
>>>> versions. It probably requires more work, but could be more robust than
>>>> parsing the version from the command line?
>>>
>>> Hm, might be an option. Parsing readelf output is pretty ugly, here I agree.
>>> In general it feels more like a binutils issue, so we have to workaround it
>>> in either way.
>>>
>>> Is Jakub or someone else working on it?
>>>
>>> Thanks!
>>>
>>
>> Jakub isn't. On our side, I noticed last week that there was this change
>> in binutils, and started to have a look at how these "features" work.
>> But I have nothing that works so far, so feel free to tackle this.
>>
>> Quentin
> 
> Hi Quentin!
> 
> Can you, please, check that the patch below works in your environment.
> 
> Thanks!


Hi Roman,

It failed for me, but I could make it work with just a small fix in
tools/build/feature/Makefile, see below. I also add some generic
comments inline.

> 
> --
> 
> 
> From b08deabf42e4c143b9e0eec8c49714e4d2c928e3 Mon Sep 17 00:00:00 2001
> From: Roman Gushchin <guro@xxxxxx>
> Date: Wed, 20 Dec 2017 13:27:32 +0000
> Subject: [RFC PATCH net-next] tools/bpftool: fix bpftool build with bintutils
>  >= 2.8
> 
> Bpftool build is broken with binutils version 2.28 and later.
> The cause is commit 003ca0fd2286 ("Refactor disassembler selection")
> in the binutils repo, which changed the disassembler() function
> signature.
> 
> Fix this by adding a new "feature" to the tools/build/features
> infrastructure and make it responsible for decision which
> disassembler() function signature to use.
> 
> Signed-off-by: Roman Gushchin <guro@xxxxxx>
> Cc: Jakub Kicinski <jakub.kicinski@xxxxxxxxxxxxx>
> Cc: Alexei Starovoitov <ast@xxxxxxxxxx>
> Cc: Daniel Borkmann <daniel@xxxxxxxxxxxxx>
> ---
>  tools/bpf/Makefile                      | 18 ++++++++++++++++++
>  tools/bpf/bpf_jit_disasm.c              |  7 +++++++
>  tools/bpf/bpftool/Makefile              | 13 +++++++++++++
>  tools/bpf/bpftool/jit_disasm.c          |  7 +++++++
>  tools/build/feature/Makefile            |  4 ++++
>  tools/build/feature/test-disassembler.c | 15 +++++++++++++++
>  6 files changed, 64 insertions(+)
>  create mode 100644 tools/build/feature/test-disassembler.c
> 
> diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
> index 07a6697466ef..c62b3a311486 100644
> --- a/tools/bpf/Makefile
> +++ b/tools/bpf/Makefile
> @@ -9,6 +9,24 @@ MAKE = make
>  CFLAGS += -Wall -O2
>  CFLAGS += -D__EXPORTED_HEADERS__ -I../../include/uapi -I../../include
>  
> +ifeq ($(srctree),)
> +srctree := $(patsubst %/,%,$(dir $(CURDIR)))
> +srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +

libbpf has a FEATURE_USER = .libbpf here, that seems to be used as a
suffix for file FEATURE-DUMP. Not sure what it is used for, but could be
a good practice, so we should check this maybe.

> +FEATURE_TESTS = disassembler
> +FEATURE_DISPLAY = disassembler

While at it, could you add the "bfd" feature verification before
"disassembler"? The former already exists in tools/build/feature.
Logically feature "disassembler" would fail to be detected if bfd is
missing, but that would seem cleaner to me.

> +
> +ifeq ($(FEATURES_DUMP),)
> +include $(srctree)/tools/build/Makefile.feature
> +else
> +include $(FEATURES_DUMP)
> +endif
> +
> +ifeq ($(feature-disassembler), 1)
> +CFLAGS += -DNEW_DISSASSEMBLER_SIGNATURE
> +endif

Nice, but this means that we check the feature for all targets. This is
not necessary, in particular in bpftool Makefile below, this means that
we check the feature even on `make doc` for example.

libbpf has something in its Makefile to prevent that, you may want to
have a look at its check_feat variable.

Also checking the feature compiles some files in the tree that are not
removed with existing "clean" target. Again, I use libbpf as a reference
(see config-clean target, and removal of FEATURE-DUMP.

> +
>  %.yacc.c: %.y
>  	$(YACC) -o $@ -d $<
>  
> diff --git a/tools/bpf/bpf_jit_disasm.c b/tools/bpf/bpf_jit_disasm.c
> index 75bf526a0168..a5f4dbacdb11 100644
> --- a/tools/bpf/bpf_jit_disasm.c
> +++ b/tools/bpf/bpf_jit_disasm.c
> @@ -72,7 +72,14 @@ static void get_asm_insns(uint8_t *image, size_t len, int opcodes)
>  
>  	disassemble_init_for_target(&info);
>  
> +#ifdef NEW_DISSASSEMBLER_SIGNATURE

Could we find another macro name? It makes perfect sense today, but
maybe no so much in the future, in particular if the prototype were to
change again. Something like BINUTILS_VERSION_GEQ_2_29, or
DISASM_FOUR_ARGS_SIGNATURE?

Also for the name of the feature ("disassembler" in your patch) we might
want to find something more explicit, we're not trying to check whether
the disassembler simply exists. "diassembler-four-args"? Any better idea?

> +	disassemble = disassembler(bfd_get_arch(bfdf),
> +				   bfd_big_endian(bfdf),
> +				   bfd_get_mach(bfdf),
> +				   bfdf);

Nit: we already have some values in info.arch and info.mach, no need to
call the functions again.

> +#else
>  	disassemble = disassembler(bfdf);
> +#endif
>  	assert(disassemble);
>  
>  	do {
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index 3f17ad317512..9c089cfa5f3f 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile

(Same comments as for previous Makefile.)

> @@ -43,6 +43,19 @@ LIBS = -lelf -lbfd -lopcodes $(LIBBPF)
>  INSTALL ?= install
>  RM ?= rm -f
>  
> +FEATURE_TESTS = disassembler
> +FEATURE_DISPLAY = disassembler
> +
> +ifeq ($(FEATURES_DUMP),)
> +include $(srctree)/tools/build/Makefile.feature
> +else
> +include $(FEATURES_DUMP)
> +endif
> +
> +ifeq ($(feature-disassembler), 1)
> +CFLAGS += -DNEW_DISSASSEMBLER_SIGNATURE
> +endif
> +
>  include $(wildcard *.d)
>  
>  all: $(OUTPUT)bpftool
> diff --git a/tools/bpf/bpftool/jit_disasm.c b/tools/bpf/bpftool/jit_disasm.c
> index 1551d3918d4c..8295e2f14ed7 100644
> --- a/tools/bpf/bpftool/jit_disasm.c
> +++ b/tools/bpf/bpftool/jit_disasm.c
> @@ -107,7 +107,14 @@ void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes)
>  
>  	disassemble_init_for_target(&info);
>  
> +#ifdef NEW_DISSASSEMBLER_SIGNATURE
> +	disassemble = disassembler(bfd_get_arch(bfdf),
> +				   bfd_big_endian(bfdf),
> +				   bfd_get_mach(bfdf),
> +				   bfdf);
> +#else
>  	disassemble = disassembler(bfdf);
> +#endif
>  	assert(disassemble);
>  
>  	if (json_output)
> diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
> index 96982640fbf8..91f937943918 100644
> --- a/tools/build/feature/Makefile
> +++ b/tools/build/feature/Makefile
> @@ -13,6 +13,7 @@ FILES=                                          \
>           test-hello.bin                         \
>           test-libaudit.bin                      \
>           test-libbfd.bin                        \
> +         test-disassembler.bin                  \
>           test-liberty.bin                       \
>           test-liberty-z.bin                     \
>           test-cplus-demangle.bin                \
> @@ -188,6 +189,9 @@ $(OUTPUT)test-libpython-version.bin:
>  $(OUTPUT)test-libbfd.bin:
>  	$(BUILD) -DPACKAGE='"perf"' -lbfd -lz -liberty -ldl
>  
> +$(OUTPUT)test-disassembler.bin:
> +	$(BUILD) -lopcodes

This is where it failed for me. Functions bfd_get_arch(), bfd_get_mach()
and bfd_big_endian() are not in lib opcodes, so I had to add `-lbfd` to
this line. With this fix, I got the feature being detected normally (or
not, if the machine still has the old version of binutils).

> +
>  $(OUTPUT)test-liberty.bin:
>  	$(CC) $(CFLAGS) -Wall -Werror -o $@ test-libbfd.c -DPACKAGE='"perf"' $(LDFLAGS) -lbfd -ldl -liberty
>  
> diff --git a/tools/build/feature/test-disassembler.c b/tools/build/feature/test-disassembler.c
> new file mode 100644
> index 000000000000..45ce65cfddf0
> --- /dev/null
> +++ b/tools/build/feature/test-disassembler.c
> @@ -0,0 +1,15 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <bfd.h>
> +#include <dis-asm.h>
> +
> +int main(void)
> +{
> +	bfd *abfd = bfd_openr(NULL, NULL);
> +
> +	disassembler(bfd_get_arch(abfd),
> +		     bfd_big_endian(abfd),
> +		     bfd_get_mach(abfd),
> +		     abfd);
> +
> +	return 0;
> +}
> 

Overall, looks like a nice fix for compiling bpftool, thanks a lot!

Quentin