Re: [PATCH] kbuild: Support clang-$ver builds

From: Sedat Dilek
Date: Sat Oct 30 2021 - 21:18:43 EST

Next message: kernel test robot: "[arnd-playground:randconfig-5.16-next 51/151] arch/x86/include/asm/ftrace.h:7:3: error: Compiler does not support fentry?"
Previous message: kernel test robot: "[tip:auto-latest] BUILD SUCCESS 95bbdb1410843e34fcecff0818ef1421286acb33"
In reply to: Peter Zijlstra: "Re: [PATCH] kbuild: Support clang-$ver builds"
Next in thread: Peter Zijlstra: "Re: [PATCH] kbuild: Support clang-$ver builds"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Sat, Oct 30, 2021 at 4:34 PM Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
>
>
> Latestest greatness... :-)
>
> ---
> Subject: kbuild: Fix clang/llvm build infra
>
> Debian (and derived) distros ship their compilers as -$ver suffixed
> binaries. For gcc it is sufficent to use:
>
> $ make CC=gcc-12
>
> However, clang builds (esp. clang-lto) need a whole array of tools to be
> exactly right, leading to unweildy stuff like:
>
> $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1
>
> which is, quite franktly, totally insane and unusable. Instead make
> the CC variable DTRT, enabling one such as myself to use:
>
> $ make CC=clang-13
>
> This also lets one quickly test different clang versions.
> Additionally, also support path based LLVM suites like:
>
> $ make CC=/opt/llvm/bin/clang
>

Hi Peter,

Thanks for bringing this up again.

Your issue is not new for Debianist and Linux-LLVM/Clang folks.

But let me comment.

I agree, it is preferable or should be treated the same way in using
clang-$ver like gcc-$ver.

Background:
Debian uses as meta-package llvm-toolchain-$ver and splits stuff into
different Debian packages like the Clang compiler. LLD linker and LLVM
(bin)utils.
These packages place their files in separate directories to
distinguish llvm-toolchain-$ver.
Binaries like clang-$ver, ld.lld-$ver, llvm-strip-$ver, etc. you find
in the /usr/bin/ directory which are symlinks.
Clang compiler is shipped with clang-$ver Debian package.
LLD linker is shipped as a separate Debian package called lld-$ver.
LLVM (bin)utils like llvm-strip-$ver are shipped with the llvm-$ver
Debian package.

Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities

EXAMPLES (here: llvm-toolchain-13):

$ dpkg -S $(which clang-13)
clang-13: /usr/bin/clang-13

$ dpkg -S /usr/bin/ld.lld-13
lld-13: /usr/bin/ld.lld-13

$ dpkg -S /usr/bin/llvm-strip-13
llvm-13: /usr/bin/llvm-strip-13

$ LC_ALL=C ll /usr/bin/clang-13
lrwxrwxrwx 1 root root 24 Oct 23 10:33 /usr/bin/clang-13 ->
../lib/llvm-13/bin/clang

$ LC_ALL=C ll /usr/bin/ld.lld-13
lrwxrwxrwx 1 root root 25 Oct 23 10:33 /usr/bin/ld.lld-13 ->
../lib/llvm-13/bin/ld.lld

$ LC_ALL=C ll /usr/bin/llvm-strip-13
lrwxrwxrwx 1 root root 29 Oct 23 10:33 /usr/bin/llvm-strip-13 ->
../lib/llvm-13/bin/llvm-strip

As a compromise I use in my build_linux-llvm-toolchain.sh script
$LLVM_TOOLCHAIN_PATH:
...
### GCC and LLVM version settings
##GCC_MAJOR_VER="10"
LLVM_MAJOR_VER="13"

### LLVM toolchain path options
# Explicitly set PATH to simplify `make LLVM=1` handling (see "LLVM
tools options")
# Option-1: Use system's LLVM toolchain
# Option-2: Use selfmade LLVM toolchain (here: ThinLTO+PGO optimized
via tc-build)
# Option-3: Set custom PATH in ~/.bashrc
##LLVM_TOOLCHAIN_PATH="/usr/lib/llvm-${LLVM_MAJOR_VER}/bin"
LLVM_TOOLCHAIN_PATH="/opt/llvm-toolchain/bin"
if [ -d ${LLVM_TOOLCHAIN_PATH} ]; then
export PATH="${LLVM_TOOLCHAIN_PATH}:${PATH}"
fi
...

But...

...for several new features like Clang-LTO we need the LLVM full "eco system".
LLVM eco system means compiler, linker and binutils.
In several talks at LPC 2021 talkers used the term "LLVM/binutils"
like for GNU/binutils.
The LLVM/Clang kernel-docs talk about "LLVM util(itie)s" which is set
by LLVM=1 or not (LLVM=0).

Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities

AFAICS your patch has a primary focus to fix the LLVM/Clang
infrastructure when CONFIG_LTO_CLANG=y, right?
Can you massage the commit message to reflect this?

IMHO your patch can be improved to check for CONFIG_LTO_CLANG=y where
LLVM=1 is mandatory and IIRC LLVM_IAS=1, too.
( The default to use LLVM/Clang-IAS (Integrated ASsembler) was changed
recently for some combinations arches + Clang-LTO? )

Before I forgot:
Can you add a comment for the PPC and S390 situation in the commit message.

Please remember people also want to use combinations like gcc-$ver and
LLVM/binutils like ld.lld-$ver.
GCC-10 + LLD-$VER was the fastest combination to build a Linux Debian
package here on my Intel SandyBridge CPU system.
( There exists overrides like LD=... STRIP=... and I use
XXX_FOR_BUILD=... (see my attached build-script). )

Thanks.

Regards,
- Sedat -

> Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
> ---
> Makefile | 45 +++++++++++++++++++++++++++---------
> tools/scripts/Makefile.include | 50 ++++++++++++++++++++++++++++-------------
> 2 files changed, 68 insertions(+), 27 deletions(-)
>
> --- a/Makefile
> +++ b/Makefile
> @@ -423,9 +423,29 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_C
> HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null)
> HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null)
>
> -ifneq ($(LLVM),)
> -HOSTCC = clang
> -HOSTCXX = clang++
> +# powerpc and s390 don't yet work with LLVM as a whole
> +ifeq ($(ARCH),powerpc)
> +LLVM = 0
> +endif
> +ifeq ($(ARCH),s390)
> +LLVM = 0
> +endif
> +
> +# otherwise, if CC=clang, default to using LLVM to enable LTO
> +CC_BASE := $(shell echo $(CC) | sed 's/.*\///')
> +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5")
> +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0)
> +LLVM ?= 1
> +LLVM_PFX := $(shell echo $(CC) | sed 's/$.*\/$\?.*/\1/')
> +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-")
> +endif
> +
> +# if not set by now, do not use LLVM
> +LLVM ?= 0
> +
> +ifneq ($(LLVM),0)
> +HOSTCC = $(LLVM_PFX)clang$(LLVM_SFX)
> +HOSTCXX = $(LLVM_PFX)clang++$(LLVM_SFX)
> else
> HOSTCC = gcc
> HOSTCXX = g++
> @@ -442,15 +462,15 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS)
>
> # Make variables (CC, etc...)
> CPP = $(CC) -E
> -ifneq ($(LLVM),)
> -CC = clang
> -LD = ld.lld
> -AR = llvm-ar
> -NM = llvm-nm
> -OBJCOPY = llvm-objcopy
> -OBJDUMP = llvm-objdump
> -READELF = llvm-readelf
> -STRIP = llvm-strip
> +ifneq ($(LLVM),0)
> +CC = $(LLVM_PFX)clang$(LLVM_SFX)
> +LD = $(LLVM_PFX)ld.lld$(LLVM_SFX)
> +AR = $(LLVM_PFX)llvm-ar$(LLVM_SFX)
> +NM = $(LLVM_PFX)llvm-nm$(LLVM_SFX)
> +OBJCOPY = $(LLVM_PFX)llvm-objcopy$(LLVM_SFX)
> +OBJDUMP = $(LLVM_PFX)llvm-objdump$(LLVM_SFX)
> +READELF = $(LLVM_PFX)llvm-readelf$(LLVM_SFX)
> +STRIP = $(LLVM_PFX)llvm-strip$(LLVM_SFX)
> else
> CC = $(CROSS_COMPILE)gcc
> LD = $(CROSS_COMPILE)ld
> @@ -461,6 +481,7 @@ OBJDUMP = $(CROSS_COMPILE)objdump
> READELF = $(CROSS_COMPILE)readelf
> STRIP = $(CROSS_COMPILE)strip
> endif
> +
> PAHOLE = pahole
> RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids
> LEX = flex
> --- a/tools/scripts/Makefile.include
> +++ b/tools/scripts/Makefile.include
> @@ -51,12 +51,32 @@ define allow-override
> $(eval $(1) = $(2)))
> endef
>
> -ifneq ($(LLVM),)
> -$(call allow-override,CC,clang)
> -$(call allow-override,AR,llvm-ar)
> -$(call allow-override,LD,ld.lld)
> -$(call allow-override,CXX,clang++)
> -$(call allow-override,STRIP,llvm-strip)
> +# powerpc and s390 don't yet work with LLVM as a whole
> +ifeq ($(ARCH),powerpc)
> +LLVM = 0
> +endif
> +ifeq ($(ARCH),s390)
> +LLVM = 0
> +endif
> +
> +# otherwise, if CC=clang, default to using LLVM to enable LTO
> +CC_BASE := $(shell echo $(CC) | sed 's/.*\///')
> +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5")
> +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0)
> +LLVM ?= 1
> +LLVM_PFX := $(shell echo $(CC) | sed 's/$.*\/$\?.*/\1/')
> +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-")
> +endif
> +
> +# if not set by now, do not use LLVM
> +LLVM ?= 0
> +
> +ifneq ($(LLVM),0)
> +$(call allow-override,CC,$(LLVM_PFX)clang$(LLVM_SFX))
> +$(call allow-override,AR,$(LLVM_PFX)llvm-ar$(LLVM_SFX))
> +$(call allow-override,LD,$(LLVM_PFX)ld.lld$(LLVM_SFX))
> +$(call allow-override,CXX,$(LLVM_PFX)clang++$(LLVM_SFX))
> +$(call allow-override,STRIP,$(LLVM_PFX)llvm-strip$(LLVM_SFX))
> else
> # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix.
> $(call allow-override,CC,$(CROSS_COMPILE)gcc)
> @@ -68,10 +88,10 @@ endif
>
> CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?)
>
> -ifneq ($(LLVM),)
> -HOSTAR ?= llvm-ar
> -HOSTCC ?= clang
> -HOSTLD ?= ld.lld
> +ifneq ($(LLVM),0)
> +HOSTAR ?= $(LLVM_PFX)llvm-ar$(LLVM_SFX)
> +HOSTCC ?= $(LLVM_PFX)clang$(LLVM_SFX)
> +HOSTLD ?= $(LLVM_PFX)ld.lld$(LLVM_SFX)
> else
> HOSTAR ?= ar
> HOSTCC ?= gcc
> @@ -79,11 +99,11 @@ HOSTLD ?= ld
> endif
>
> # Some tools require Clang, LLC and/or LLVM utils
> -CLANG ?= clang
> -LLC ?= llc
> -LLVM_CONFIG ?= llvm-config
> -LLVM_OBJCOPY ?= llvm-objcopy
> -LLVM_STRIP ?= llvm-strip
> +CLANG ?= $(LLVM_PFX)clang$(LLVM_SFX)
> +LLC ?= $(LLVM_PFX)llc$(LLVM_SFX)
> +LLVM_CONFIG ?= $(LLVM_PFX)llvm-config$(LLVM_SFX)
> +LLVM_OBJCOPY ?= $(LLVM_PFX)llvm-objcopy$(LLVM_SFX)
> +LLVM_STRIP ?= $(LLVM_PFX)llvm-strip$(LLVM_SFX)
>
> ifeq ($(CC_NO_CLANG), 1)
> EXTRA_WARNINGS += -Wstrict-aliasing=3

Attachment: build_linux-llvm-toolchain.sh
Description: application/shellscript

Next message: kernel test robot: "[arnd-playground:randconfig-5.16-next 51/151] arch/x86/include/asm/ftrace.h:7:3: error: Compiler does not support fentry?"
Previous message: kernel test robot: "[tip:auto-latest] BUILD SUCCESS 95bbdb1410843e34fcecff0818ef1421286acb33"
In reply to: Peter Zijlstra: "Re: [PATCH] kbuild: Support clang-$ver builds"
Next in thread: Peter Zijlstra: "Re: [PATCH] kbuild: Support clang-$ver builds"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]