Re: Clang patch stacks for LTS kernels (v4.4 and v4.9) and status update

From: Sedat Dilek
Date: Tue Apr 24 2018 - 07:54:51 EST

Hi Matthias,

a big thank you for giving all the informations!

I used your mka/llvm/v4.14_ext Git tree...

https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/sandbox/mka/llvm/v4.14_ext

...and was able to compile with clang-6.0 from Debian/buster64 with...

...reverting the clang-3/clang-4 patches...

user$ for p in 69e44656ae43 222b88977a00 cdfcf1e45537 26f14c9225a6
0385a18e9995 68dab143c9b4 ; do echo [ $p ] ; LC_ALL=C git revert
--no-edit $p ; done

0001-Revert-CLANG3-core-clang-work-around-x86-regparm-int.patch
0002-Revert-CLANG4-futex-don-t-optimize-futex_detect_cmpx.patch
0003-Revert-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
0004-Revert-CLANG4-arm64-prefetch-Use-__builtin_arm_prefe.patch
0005-Revert-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-para.patch
0006-Revert-CLANG4-crypto-arm64-aes-ce-Explicitly-pass-th.patch

...and needed two additional patches from upstream:

0007-kbuild-clang-remove-crufty-HOSTCFLAGS.patch <--- Label with
BACKPORT (XXX: Backported; Required when using HOSTCC in make-line,
see below)
0008-x86-xen-remove-the-use-of-VLAIS.patch <--- Label with UPSTREAM
(XXX: cherry-picked)

These 3 patches in your Git branch are in Linux v4.14.36...

9af5ddf981ed BACKPORT: kbuild: disable clang's default use of
-fmerge-all-constants
f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
cross compiling with Clang

...and can be dropped.
I do not know your work-flow/policy: Maybe you want to keep your
patch-stack against vanilla Linux v4.14 - without stables from
linux-stable-4.14.y.

For easy switching "mycompiler" I use a wrapper-script:

root# cat /usr/bin/mycompiler
#!/bin/bash

clang-6.0 "$@"
- EOF -

user$ cp -v /boot/config-4.14.35-1-iniza-amd64 .config
user$ MAKE="make V=1" ; COMPILER="mycompiler" ;
MAKE_OPTS="CC=$COMPILER HOSTCC=$COMPILER"
user$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
silentoldconfig < /dev/null

The diffconfig to my current kernel-config-4.14 looks like this...

user$ ./scripts/diffconfig /boot/config-4.14.35-1-iniza-amd64 .config
-ARCH_HAS_REFCOUNT y
-BPF_JIT_ALWAYS_ON n
-EXOFS_DEBUG n
-EXOFS_FS m
-GENERIC_CPU_VULNERABILITIES y
-KASAN n
-ORE m
-PAGE_TABLE_ISOLATION y
-RETPOLINE y
-UNWINDER_FRAME_POINTER n
-UNWINDER_GUESS n
-UNWINDER_ORC y
+FRAME_POINTER y
+FRAME_POINTER_UNWINDER y
+GUESS_UNWINDER n
+HAVE_ARCH_KMEMCHECK y
+HAVE_RELIABLE_STACKTRACE y
+ORC_UNWINDER n

Unfortunately, I cannot boot into the generated kernel on bare metal.

Checking with QEMU (version: 2.12~rc3) and catching earlyprintk, I see this...

user$ echo $KPATH
$HOME/src/linux-kernel/important-files

user$ ls -al
insgesamt 344916
drwxr-xr-x 2 sdi sdi 4096 Apr 24 13:15 .
drwxr-xr-x 20 sdi sdi 4096 Apr 24 13:13 ..
-rw-r--r-- 1 sdi sdi 4528416 Apr 24 12:42 bzImage
lrwxrwxrwx 1 sdi sdi 35 Apr 24 13:15 initrd.img ->
initrd.img-4.14.0-1-iniza-llvmlinux
-rw-r--r-- 1 sdi sdi 25572955 Apr 24 13:08
initrd.img-4.14.0-1-iniza-llvmlinux
-rw-r--r-- 1 sdi sdi 2887195 Apr 24 12:42 System.map
-rwxr-xr-x 1 sdi sdi 326116744 Apr 24 12:42 vmlinux

user$ sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
-initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
"root=/dev/ram0 console=ttyS0 hung_task_panic=1
earlyprintk=ttyS0,115200"

Probing EDD (edd=off to disable)... ok
[ 0.000000] Linux version 4.14.0-1-iniza-llvmlinux
(sedat.dilek@xxxxxxxxx@iniza) (clang version 6.0.0-1
(tags/RELEASE_600/final)) #1 SMP Tue Apr 24 12:42:21 CEST 2018
[ 0.000000] Command line: root=/dev/ram0 console=ttyS0
hung_task_panic=1 earlyprintk=ttyS0,115200
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
[ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] bootconsole [earlyser0] enabled
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] random: fast init done
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.11.1-1 04/01/2014
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
[ 0.000000] x86/PAT: PAT not supported by CPU.
[ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
Memory KASLR using RDTSC...
[ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
mapped at [ffffffffff000d60]
[ 0.000000] RAMDISK: [mem 0x1e77c000-0x1ffdffff]
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
[ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
BXPCHPET 00000001 BXPC 00000001)
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
[ 0.000000] NODE_DATA(0) allocated [mem 0x1e777000-0x1e77bfff]
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 0:1e76f001, primary cpu clock
[ 0.000000] kvm-clock: using sched offset of 528742140 cycles
[ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
[ 0.000000] Normal empty
[ 0.000000] Device empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on KVM
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
nr_cpu_ids:1 nr_node_ids:1
[ 0.000000] percpu: Embedded 37 pages/cpu @ffff96dd9e400000 s114392
r8192 d28968 u2097152
[ 0.000000] KVM setup async PF for cpu 0
[ 0.000000] kvm-stealtime: cpu 0, msr 1e40d900
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
hung_task_panic=1 earlyprintk=ttyS0,115200
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Memory: 474480K/523768K available (7639K kernel code,
1005K rwdata, 2936K rodata, 1636K init, 688K bss, 49288K reserved, 0K
cma-reserved)
[ 0.000000] ftrace: allocating 27586 entries in 108 pages
[ 0.004000] Hierarchical RCU implementation.
[ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
[ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] console [ttyS0] enabled
[ 0.004000] console [ttyS0] enabled
[ 0.004000] bootconsole [earlyser0] disabled
[ 0.004000] bootconsole [earlyser0] disabled
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 19112604467 ns
[ 0.004000] general protection fault: 0000 [#1] SMP
[ 0.004000] Modules linked in:
[ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
4.14.0-1-iniza-llvmlinux #1
[ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.11.1-1 04/01/2014
[ 0.004000] task: ffffffffaa610480 task.stack: ffffffffaa600000
[ 0.004000] RIP: 0010:irq_work_tick+0x9d/0x110
[ 0.004000] RSP: 0000:ffff96dd9e403e58 EFLAGS: 00010046
[ 0.004000] RAX: 0000000000000082 RBX: ffff96dd9e411d80 RCX: adecc9cc04e2ca00
[ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96dd9e41ba38
[ 0.004000] RBP: ffff96dd9e403e78 R08: 0000000000000000 R09: 0000000000000018
[ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffaa61bd40
[ 0.004000] R13: 0000000000000000 R14: ffffffffaa610480 R15: 0000000000000000
[ 0.004000] FS: 0000000000000000(0000) GS:ffff96dd9e400000(0000)
knlGS:0000000000000000
[ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.004000] CR2: 00000000ffffffff CR3: 0000000008a09000 CR4: 00000000000006b0
[ 0.004000] Call Trace:
[ 0.004000] <IRQ>
[ 0.004000] update_process_times+0x6e/0xa0
[ 0.004000] tick_periodic+0x78/0x90
[ 0.004000] tick_handle_periodic+0x26/0x80
[ 0.004000] timer_interrupt+0x13/0x20
[ 0.004000] __handle_irq_event_percpu+0x106/0x230
[ 0.004000] handle_irq_event+0x5a/0xc0
[ 0.004000] handle_level_irq+0x11a/0x190
[ 0.004000] handle_irq+0x1f/0x30
[ 0.004000] do_IRQ+0x4b/0xd0
[ 0.004000] common_interrupt+0x93/0x93
[ 0.004000] </IRQ>
[ 0.004000] RIP: 0010:native_restore_fl+0x12/0x20
[ 0.004000] RSP: 0000:ffffffffaa603e10 EFLAGS: 00000286 ORIG_RAX:
ffffffffffffffcf
[ 0.004000] RAX: 0000000000000001 RBX: ffff96dd9d0c0608 RCX: 0000000000000000
[ 0.004000] RDX: ffff96dd9d0c0400 RSI: 0000000000000286 RDI: 0000000000000286
[ 0.004000] RBP: ffffffffaa603e18 R08: 0000000000000001 R09: 000000000000003f
[ 0.004000] R10: 0000000000000286 R11: 0000000000000007 R12: ffff96dd9d0c0514
[ 0.004000] R13: ffff96dd9d0c04e0 R14: ffffffffaa61bd40 R15: ffff96dd9d0c0400
[ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
[ 0.004000] __setup_irq+0x5d9/0x780
[ 0.004000] setup_irq+0x5c/0x90
[ 0.004000] hpet_time_init+0x32/0x40
[ 0.004000] x86_late_time_init+0x10/0x20
[ 0.004000] start_kernel+0x45d/0x580
[ 0.004000] x86_64_start_kernel+0x30f/0x320
[ 0.004000] secondary_startup_64+0xa5/0xa5
[ 0.004000] Code: f0 f0 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48
c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 62 aa f6
c4 02 75 4d <48> 83 39 00 74 3e 31 db 48 87 19 48 85 db 74 34 0f 1f 00
48 8d
[ 0.004000] RIP: irq_work_tick+0x9d/0x110 RSP: ffff96dd9e403e58
[ 0.004000] ---[ end trace 8c7851007fbc6b6f ]---
[ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
[ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

Is this saying someting to you?

I am attaching my kernel-config file and the two additional patches.

Do you need further informations?

Thanks.

Regards,
- Sedat -

Attachment: config-4.14.0-1-iniza-llvmlinux
Description: Binary data

From 47844fdbe3998ef477201033acd0eff615da7cbf Mon Sep 17 00:00:00 2001
From: Nick Desaulniers <nick.desaulniers@xxxxxxxxx>
Date: Sat, 7 Oct 2017 13:23:23 -0700
Subject: [PATCH 7/8] kbuild: clang: remove crufty HOSTCFLAGS

When compiling with `make CC=clang HOSTCC=clang`, I was seeing warnings
that clang did not recognize -fno-delete-null-pointer-checks for HOSTCC
targets. These were added in commit 61163efae020 ("kbuild: LLVMLinux:
Add Kbuild support for building kernel with Clang").

Clang does not support -fno-delete-null-pointer-checks, so adding it to
HOSTCFLAGS if HOSTCC is clang does not make sense.

It's not clear why the other warnings were disabled, and just for
HOSTCFLAGS, but I can remove them, add -Werror to HOSTCFLAGS and compile
with clang just fine.

Suggested-by: Masahiro Yamada <yamada.masahiro@xxxxxxxxxxxxx>
Signed-off-by: Nick Desaulniers <nick.desaulniers@xxxxxxxxx>
Signed-off-by: Masahiro Yamada <yamada.masahiro@xxxxxxxxxxxxx>
[ Backport to Linux v4.14 LTS ]
Signed-off-by: Sedat Dilek <sedat.dilek@xxxxxxxxx>
---
Makefile | 5 -----
1 file changed, 5 deletions(-)

diff --git a/Makefile b/Makefile
index 4df63732cc4e..8d045052c080 100644
--- a/Makefile
+++ b/Makefile
@@ -307,11 +307,6 @@ HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS)
HOSTLDFLAGS := $(HOST_LFS_LDFLAGS)
HOST_LOADLIBES := $(HOST_LFS_LIBS)

-ifeq ($(shell $(HOSTCC) -v 2>&1 | grep -c "clang version"), 1)
-HOSTCFLAGS += -Wno-unused-value -Wno-unused-parameter \
- -Wno-missing-field-initializers -fno-delete-null-pointer-checks
-endif
-
# Decide whether to build built-in, modular, or both.
# Normally, just do built-in.

--
2.17.0

From c3d77d404c2fafe14f0e36745ef60509fe84c959 Mon Sep 17 00:00:00 2001
From: Nick Desaulniers <nick.desaulniers@xxxxxxxxx>
Date: Sat, 6 Jan 2018 13:39:48 -0800
Subject: [PATCH 8/8] x86: xen: remove the use of VLAIS

Variable Length Arrays In Structs (VLAIS) is not supported by Clang, and
frowned upon by others.

https://lkml.org/lkml/2013/9/23/500

Here, the VLAIS was used because the size of the bitmap returned from
xen_mc_entry() depended on possibly (based on kernel configuration)
runtime sized data. Rather than declaring args as a VLAIS then calling
sizeof on *args, we calculate the appropriate sizeof args manually.
Further, we can get rid of the #ifdef's and rely on num_possible_cpus()
(thanks to a helpful checkpatch warning from an earlier version of this
patch).

Suggested-by: Juergen Gross <jgross@xxxxxxxx>
Signed-off-by: Nick Desaulniers <nick.desaulniers@xxxxxxxxx>
Reviewed-by: Juergen Gross <jgross@xxxxxxxx>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>
---
arch/x86/xen/mmu_pv.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 71495f1a86d7..1a2f4e653c6b 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1339,20 +1339,18 @@ static void xen_flush_tlb_others(const struct cpumask *cpus,
{
struct {
struct mmuext_op op;
-#ifdef CONFIG_SMP
- DECLARE_BITMAP(mask, num_processors);
-#else
DECLARE_BITMAP(mask, NR_CPUS);
-#endif
} *args;
struct multicall_space mcs;
+ const size_t mc_entry_size = sizeof(args->op) +
+ sizeof(args->mask[0]) * BITS_TO_LONGS(num_possible_cpus());

trace_xen_mmu_flush_tlb_others(cpus, info->mm, info->start, info->end);

if (cpumask_empty(cpus))
return; /* nothing to do */

- mcs = xen_mc_entry(sizeof(*args));
+ mcs = xen_mc_entry(mc_entry_size);
args = mcs.args;
args->op.arg2.vcpumask = to_cpumask(args->mask);

--
2.17.0