Re: [RFC v2 1/2] perf: support build BPF skeletons with perf

[Jiri Olsa]

On Mon, Nov 30, 2020 at 11:36:46PM -0800, Song Liu wrote:
> BPF programs are useful in perf to profile BPF programs. BPF skeleton is
> by far the easiest way to write BPF tools. Enable building BPF skeletons
> in util/bpf_skel. A dummy bpf skeleton is added. More bpf skeletons will
> be added for different use cases.
> 
> Signed-off-by: Song Liu <songliubraving@xxxxxx>
> ---
>  tools/bpf/bpftool/Makefile           |  2 ++
>  tools/build/Makefile.feature         |  3 +-
>  tools/perf/Makefile.config           |  9 +++++
>  tools/perf/Makefile.perf             | 52 ++++++++++++++++++++++++++--
>  tools/perf/util/bpf_skel/.gitignore  |  3 ++
>  tools/perf/util/bpf_skel/dummy.bpf.c | 19 ++++++++++
>  tools/scripts/Makefile.include       |  1 +
>  7 files changed, 86 insertions(+), 3 deletions(-)
>  create mode 100644 tools/perf/util/bpf_skel/.gitignore
>  create mode 100644 tools/perf/util/bpf_skel/dummy.bpf.c
> 
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index f60e6ad3a1dff..a01407ec78dc5 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -120,6 +120,8 @@ endif
>  
>  BPFTOOL_BOOTSTRAP := $(if $(OUTPUT),$(OUTPUT)bpftool-bootstrap,./bpftool-bootstrap)
>  
> +bootstrap: $(BPFTOOL_BOOTSTRAP)
> +
>  BOOTSTRAP_OBJS = $(addprefix $(OUTPUT),main.o common.o json_writer.o gen.o btf.o)
>  OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
>  
> diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
> index 97cbfb31b7625..95a58b5564218 100644
> --- a/tools/build/Makefile.feature
> +++ b/tools/build/Makefile.feature
> @@ -70,7 +70,8 @@ FEATURE_TESTS_BASIC :=                  \
>          libaio				\
>          libzstd				\
>          disassembler-four-args		\
> -        file-handle
> +        file-handle			\
> +        clang-bpf-co-re

this needs to go under FEATURE_TESTS_EXTRA and you need to call
$(call feature_check,clang-bpf-co-re) before checking on $(feature-clang-bpf-co-re)

otherwise the test-all will mark it as detected even if it's not

>  
>  # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list
>  # of all feature tests
> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
> index ce8516e4de34f..cb0cf06e0bb43 100644
> --- a/tools/perf/Makefile.config
> +++ b/tools/perf/Makefile.config
> @@ -621,6 +621,15 @@ ifndef NO_LIBBPF
>    endif
>  endif
>  
> +ifeq ($(feature-clang-bpf-co-re), 0)
> +  BUILD_BPF_SKEL := 0
> +endif
> +
> +ifeq ($(BUILD_BPF_SKEL), 1)
> +  $(call detected,CONFIG_PERF_BPF_SKEL)
> +  CFLAGS += -DBUILD_BPF_SKEL
> +endif
> +
>  dwarf-post-unwind := 1
>  dwarf-post-unwind-text := BUG
>  
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 7ce3f2e8b9c74..37b7ffe1db27c 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -126,6 +126,8 @@ include ../scripts/utilities.mak
>  #
>  # Define NO_LIBDEBUGINFOD if you do not want support debuginfod
>  #
> +# Define BUILD_BPF_SKEL to enable BPF skeletons
> +#
>  
>  # As per kernel Makefile, avoid funny character set dependencies
>  unexport LC_ALL
> @@ -735,7 +737,8 @@ prepare: $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)common-cmds.h archheaders $(drm_ioc
>  	$(x86_arch_prctl_code_array) \
>  	$(rename_flags_array) \
>  	$(arch_errno_name_array) \
> -	$(sync_file_range_arrays)
> +	$(sync_file_range_arrays) \
> +	bpf-skel

I think the 'prepare' target is misused already with other stuff,
there's generated bpf_counter.c dependency on util/bpf_skel/bpf_prog_profiler.skel.h
in util/.bpf_counter.o.cmd, that should triger the build no?

we have few other dependencies like that for flex/bison generation
perhaps we could follow also that 

>  
>  $(OUTPUT)%.o: %.c prepare FORCE
>  	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
> @@ -1008,7 +1011,52 @@ config-clean:
>  python-clean:
>  	$(python-clean)
>  
> -clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean config-clean fixdep-clean python-clean
> +SKEL_OUT := $(abspath util/bpf_skel)
> +SKEL_TMP_OUT := $(abspath util/bpf_skel/.tmp)
> +SKELETONS := $(SKEL_OUT)/dummy.skel.h
> +
> +ifdef BUILD_BPF_SKEL
> +CLANG ?= clang
> +LLVM_STRIP ?= llvm-strip

please move this up where we have similar setup like HOSTCC and others

> +BPFTOOL_CFLAGS := $(filter-out -D_GNU_SOURCE,$(CFLAGS))
> +BPFTOOL := $(SKEL_TMP_OUT)/bpftool-bootstrap
> +LIBBPF_SRC := $(abspath ../lib/bpf)
> +BPFOBJ := $(SKEL_TMP_OUT)/libbpf.a

we already build libbpf.a, chekc 'LIBBPF = $(BPF_PATH)libbpf.a' above

> +BPF_INCLUDE := $(SKEL_TMP_OUT)
> +submake_extras := feature_display=0
> +
> +$(SKEL_TMP_OUT):
> +	$(Q)$(MKDIR) -p $@
> +
> +$(BPFTOOL): | $(SKEL_TMP_OUT)
> +	CFLAGS= $(MAKE) $(submake_extras) -C ../bpf/bpftool \
> +		OUTPUT=$(SKEL_TMP_OUT)/ bootstrap
> +
> +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(BPFOBJ) | $(SKEL_TMP_OUT)
> +	$(call QUIET_CLANG, $@)
> +	$(Q)$(CLANG) -g -O2 -target bpf	-c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ && \
> +	$(LLVM_STRIP) -g $@
> +
> +$(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL)
> +	$(call QUIET_GENSKEL, $@)

is there a reason to use call in here? you could define QUIET_GENSKEL
to use $@ and use it the same way as we use the rest of QUIET_* stuf
in Makefile.perf

> new file mode 100644
> index 0000000000000..5263e9e6c5d83
> --- /dev/null
> +++ b/tools/perf/util/bpf_skel/.gitignore
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +.tmp
> +*.skel.h
> \ No newline at end of file
> diff --git a/tools/perf/util/bpf_skel/dummy.bpf.c b/tools/perf/util/bpf_skel/dummy.bpf.c
> new file mode 100644
> index 0000000000000..492a43a851deb
> --- /dev/null
> +++ b/tools/perf/util/bpf_skel/dummy.bpf.c

hum, what's the reason for dummy skeleton? it just adds
time to compilation no?

thanks,
jirka