Re: [PATCH v7 0/4] kexec: Fix kexec_file_load for llvm16 with PGO
From: Song Liu
Date: Thu Sep 07 2023 - 19:09:14 EST
Hi Ricardo and folks,
On Fri, May 19, 2023 at 7:48 AM Ricardo Ribalda <ribalda@xxxxxxxxxxxx> wrote:
>
> When upreving llvm I realised that kexec stopped working on my test
> platform.
>
> The reason seems to be that due to PGO there are multiple .text sections
> on the purgatory, and kexec does not supports that.
>
> Signed-off-by: Ricardo Ribalda <ribalda@xxxxxxxxxxxx>
We are seeing WARNINGs like the following while kexec'ing a PGO and
LTO enabled kernel:
WARNING: CPU: 26 PID: 110894 at kernel/kexec_file.c:919
kexec_load_purgatory+0x37f/0x390
AFAICT, the warning was added by this set, and it was triggered when
we have many .text sections
in purgatory.ro. The kexec was actually successful. So I wonder
whether we really need the
WARNING here. If we disable LTO (PGO is still enabled), we don't see
the WARNING any more.
I also tested an older kernel (5.19 based), where we also see many
.text sections with LTO. It
kexec()'ed fine. (It doesn't have the WARN_ON() in
kexec_purgatory_setup_sechdrs).
Please help us fix this properly (as I really don't know much about kexec).
Thanks in advance,
Song
Here is readelf -S output on purgatory.ro.
With LTO:
readelf -W -S purgatory.ro
There are 48 section headers, starting at offset 0x4a10:
Section Headers:
[Nr] Name Type Address Off Size
ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000
000000 00 0 0 0
[ 1] .text PROGBITS 0000000000000000 000040
0000d0 00 AX 0 0 16
[ 2] .data PROGBITS 0000000000000000 001000
001000 00 WA 0 0 4096
[ 3] .rela.text RELA 0000000000000000 003788
000228 18 I 45 1 8
[ 4] .rodata PROGBITS 0000000000000000 002000
0000e0 00 A 0 0 16
[ 5] .rela.rodata RELA 0000000000000000 0039b0
000030 18 I 45 4 8
[ 6] .bss NOBITS 0000000000000000 0020e0
001000 00 WA 0 0 4096
[ 7] .text.purgatory PROGBITS 0000000000000000 0020e0
0000df 00 AX 0 0 16
[ 8] .rela.text.purgatory RELA 0000000000000000 0039e0
000060 18 I 45 7 8
[ 9] .text.warn PROGBITS 0000000000000000 0021c0
000001 00 AX 0 0 16
[10] .kexec-purgatory PROGBITS 0000000000000000 0021d0
000120 00 WA 0 0 16
[11] .comment PROGBITS 0000000000000000 003a40
000046 01 MS 0 0 1
[12] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 003a86
000005 00 E 0 0 1
[13] .text.sha256_update PROGBITS 0000000000000000 0022f0
0008eb 00 AX 0 0 16
[14] .rela.text.sha256_update RELA 0000000000000000
003a90 000060 18 I 45 13 8
[15] .text.sha224_update PROGBITS 0000000000000000 002be0
00000c 00 AX 0 0 16
[16] .rela.text.sha224_update RELA 0000000000000000
003af0 000018 18 I 45 15 8
[17] .text.sha256_final PROGBITS 0000000000000000 002bf0
0000cd 00 AX 0 0 16
[18] .rela.text.sha256_final RELA 0000000000000000 003b08
000030 18 I 45 17 8
[19] .text.sha224_final PROGBITS 0000000000000000 002cc0
0000bd 00 AX 0 0 16
[20] .rela.text.sha224_final RELA 0000000000000000 003b38
000030 18 I 45 19 8
[21] .text.sha256 PROGBITS 0000000000000000 002d80
00011d 00 AX 0 0 16
[22] .rela.text.sha256 RELA 0000000000000000 003b68
000030 18 I 45 21 8
[23] .modinfo PROGBITS 0000000000000000 002e9d
000039 00 A 0 0 1
[24] .rodata.SHA256_K PROGBITS 0000000000000000 002ee0
000100 00 A 0 0 16
[25] .rodata.__sha256_final.padding PROGBITS 0000000000000000
002fe0 000040 00 A 0 0 16
[26] .text.memcmp PROGBITS 0000000000000000 003020
00000b 00 AX 0 0 16
[27] .text.bcmp PROGBITS 0000000000000000 003030
00000b 00 AX 0 0 16
[28] .text.strcmp PROGBITS 0000000000000000 003040
000041 00 AX 0 0 16
[29] .text.strncmp PROGBITS 0000000000000000 003090
00003a 00 AX 0 0 16
[30] .text.strnlen PROGBITS 0000000000000000 0030d0
000039 00 AX 0 0 16
[31] .text.atou PROGBITS 0000000000000000 003110
000035 00 AX 0 0 16
[32] .text.simple_strtoull PROGBITS 0000000000000000 003150
0000b6 00 AX 0 0 16
[33] .text.simple_strtol PROGBITS 0000000000000000 003210
0001b6 00 AX 0 0 16
[34] .text.strlen PROGBITS 0000000000000000 0033d0
00001c 00 AX 0 0 16
[35] .text.strstr PROGBITS 0000000000000000 0033f0
00005f 00 AX 0 0 16
[36] .text.strchr PROGBITS 0000000000000000 003450
000022 00 AX 0 0 16
[37] .text.kstrtoull PROGBITS 0000000000000000 003480
000142 00 AX 0 0 16
[38] .text.boot_kstrtoul PROGBITS 0000000000000000 0035d0
00000c 00 AX 0 0 16
[39] .rela.text.boot_kstrtoul RELA 0000000000000000
003b98 000018 18 I 45 38 8
[40] .text.memset PROGBITS 0000000000000000 0035e0
00001f 00 AX 0 0 16
[41] .text.memmove PROGBITS 0000000000000000 003600
0000a6 00 AX 0 0 16
[42] .text.memcpy PROGBITS 0000000000000000 0036b0
0000a6 00 AX 0 0 16
[43] .rodata.str1.1 PROGBITS 0000000000000000 003756
000032 01 AMS 0 0 1
[44] .note.GNU-stack PROGBITS 0000000000000000 003bb0
000000 00 0 0 1
[45] .symtab SYMTAB 0000000000000000 003bb0
000948 18 47 68 8
[46] .shstrtab STRTAB 0000000000000000 0044f8
0002cd 00 0 0 1
[47] .strtab STRTAB 0000000000000000 0047c5
000248 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
Without LTO:
readelf -W -S purgatory.ro
There are 16 section headers, starting at offset 0x4130:
Section Headers:
[Nr] Name Type Address Off Size
ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000
000000 00 0 0 0
[ 1] .text PROGBITS 0000000000000000 000040
00131b 00 AX 0 0 16
[ 2] .rela.text RELA 0000000000000000 003290
000480 18 I 13 1 8
[ 3] .kexec-purgatory PROGBITS 0000000000000000 001360
000120 00 WA 0 0 16
[ 4] .comment PROGBITS 0000000000000000 003710
000046 01 MS 0 0 1
[ 5] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 003756
000005 00 E 0 0 1
[ 6] .data PROGBITS 0000000000000000 002000
001000 00 WA 0 0 4096
[ 7] .rodata PROGBITS 0000000000000000 003000
000220 00 A 0 0 16
[ 8] .rela.rodata RELA 0000000000000000 003760
000030 18 I 13 7 8
[ 9] .bss NOBITS 0000000000000000 003220
001000 00 WA 0 0 4096
[10] .modinfo PROGBITS 0000000000000000 003220
000039 00 A 0 0 1
[11] .rodata.str1.1 PROGBITS 0000000000000000 003259
000032 01 AMS 0 0 1
[12] .note.GNU-stack PROGBITS 0000000000000000 003790
000000 00 0 0 1
[13] .symtab SYMTAB 0000000000000000 003790
0006d8 18 15 43 8
[14] .shstrtab STRTAB 0000000000000000 003e68
00009c 00 0 0 1
[15] .strtab STRTAB 0000000000000000 003f04
00022b 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
[...]
> Ricardo Ribalda (4):
> kexec: Support purgatories with .text.hot sections
> x86/purgatory: Remove PGO flags
> powerpc/purgatory: Remove PGO flags
> riscv/purgatory: Remove PGO flags
>
> arch/powerpc/purgatory/Makefile | 5 +++++
> arch/riscv/purgatory/Makefile | 5 +++++
> arch/x86/purgatory/Makefile | 5 +++++
> kernel/kexec_file.c | 14 +++++++++++++-
> 4 files changed, 28 insertions(+), 1 deletion(-)
> ---
> base-commit: 58390c8ce1bddb6c623f62e7ed36383e7fa5c02f
> change-id: 20230321-kexec_clang16-4510c23d129c
>
> Best regards,
> --
> Ricardo Ribalda Delgado <ribalda@xxxxxxxxxxxx>
>