Re: [RFC][PATCH] x86: Add straight-line-speculation mitigation

From: Andrew Cooper
Date: Thu Oct 28 2021 - 16:06:48 EST

On 28/10/2021 12:44, Peter Zijlstra wrote:
> arch/x86/Kconfig | 12 +++++
> arch/x86/Makefile | 4 +
> arch/x86/crypto/aegis128-aesni-asm.S | 48 +++++++++++-----------
> arch/x86/crypto/aes_ctrby8_avx-x86_64.S | 2
> arch/x86/crypto/aesni-intel_asm.S | 56 +++++++++++++-------------
> arch/x86/crypto/aesni-intel_avx-x86_64.S | 40 +++++++++---------
> arch/x86/crypto/blake2s-core.S | 4 -
> arch/x86/crypto/blowfish-x86_64-asm_64.S | 12 ++---
> arch/x86/crypto/camellia-aesni-avx-asm_64.S | 14 +++---
> arch/x86/crypto/camellia-aesni-avx2-asm_64.S | 14 +++---
> arch/x86/crypto/camellia-x86_64-asm_64.S | 12 ++---
> arch/x86/crypto/cast5-avx-x86_64-asm_64.S | 12 ++---
> arch/x86/crypto/cast6-avx-x86_64-asm_64.S | 10 ++--
> arch/x86/crypto/chacha-avx2-x86_64.S | 6 +-
> arch/x86/crypto/chacha-avx512vl-x86_64.S | 6 +-
> arch/x86/crypto/chacha-ssse3-x86_64.S | 8 +--
> arch/x86/crypto/crc32-pclmul_asm.S | 2
> arch/x86/crypto/crc32c-pcl-intel-asm_64.S | 2
> arch/x86/crypto/crct10dif-pcl-asm_64.S | 2
> arch/x86/crypto/des3_ede-asm_64.S | 4 -
> arch/x86/crypto/ghash-clmulni-intel_asm.S | 6 +-
> arch/x86/crypto/nh-avx2-x86_64.S | 2
> arch/x86/crypto/nh-sse2-x86_64.S | 2
> arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 38 ++++++++---------
> arch/x86/crypto/serpent-avx-x86_64-asm_64.S | 10 ++--
> arch/x86/crypto/serpent-avx2-asm_64.S | 10 ++--
> arch/x86/crypto/serpent-sse2-i586-asm_32.S | 6 +-
> arch/x86/crypto/serpent-sse2-x86_64-asm_64.S | 6 +-
> arch/x86/crypto/sha1_avx2_x86_64_asm.S | 2
> arch/x86/crypto/sha1_ni_asm.S | 2
> arch/x86/crypto/sha1_ssse3_asm.S | 2
> arch/x86/crypto/sha256-avx-asm.S | 2
> arch/x86/crypto/sha256-avx2-asm.S | 2
> arch/x86/crypto/sha256-ssse3-asm.S | 2
> arch/x86/crypto/sha256_ni_asm.S | 2
> arch/x86/crypto/sha512-avx-asm.S | 2
> arch/x86/crypto/sha512-avx2-asm.S | 2
> arch/x86/crypto/sha512-ssse3-asm.S | 2
> arch/x86/crypto/sm4-aesni-avx-asm_64.S | 12 ++---
> arch/x86/crypto/sm4-aesni-avx2-asm_64.S | 8 +--
> arch/x86/crypto/twofish-avx-x86_64-asm_64.S | 10 ++--
> arch/x86/crypto/twofish-i586-asm_32.S | 4 -
> arch/x86/crypto/twofish-x86_64-asm_64-3way.S | 6 +-
> arch/x86/crypto/twofish-x86_64-asm_64.S | 4 -
> arch/x86/entry/entry_32.S | 2
> arch/x86/entry/entry_64.S | 12 ++---
> arch/x86/entry/thunk_32.S | 2
> arch/x86/entry/thunk_64.S | 2
> arch/x86/entry/vdso/vdso32/system_call.S | 2
> arch/x86/entry/vdso/vsgx.S | 2
> arch/x86/entry/vsyscall/vsyscall_emu_64.S | 6 +-
> arch/x86/include/asm/linkage.h | 16 +++++++
> arch/x86/include/asm/paravirt.h | 2
> arch/x86/include/asm/qspinlock_paravirt.h | 4 -
> arch/x86/include/asm/static_call.h | 2
> arch/x86/kernel/acpi/wakeup_32.S | 6 +-
> arch/x86/kernel/acpi/wakeup_64.S | 1
> arch/x86/kernel/alternative.c | 2
> arch/x86/kernel/ftrace.c | 2
> arch/x86/kernel/ftrace_32.S | 6 +-
> arch/x86/kernel/ftrace_64.S | 10 ++--
> arch/x86/kernel/head_32.S | 2
> arch/x86/kernel/head_64.S | 1
> arch/x86/kernel/irqflags.S | 2
> arch/x86/kernel/paravirt.c | 2
> arch/x86/kernel/relocate_kernel_32.S | 10 ++--
> arch/x86/kernel/relocate_kernel_64.S | 10 ++--
> arch/x86/kernel/sev_verify_cbit.S | 2
> arch/x86/kernel/static_call.c | 5 +-
> arch/x86/kernel/verify_cpu.S | 4 -
> arch/x86/kvm/emulate.c | 4 -
> arch/x86/kvm/svm/vmenter.S | 4 -
> arch/x86/kvm/vmx/vmenter.S | 14 +++---
> arch/x86/lib/atomic64_386_32.S | 2
> arch/x86/lib/atomic64_cx8_32.S | 16 +++----
> arch/x86/lib/checksum_32.S | 8 +--
> arch/x86/lib/clear_page_64.S | 6 +-
> arch/x86/lib/cmpxchg16b_emu.S | 4 -
> arch/x86/lib/cmpxchg8b_emu.S | 4 -
> arch/x86/lib/copy_mc_64.S | 6 +-
> arch/x86/lib/copy_page_64.S | 4 -
> arch/x86/lib/copy_user_64.S | 10 ++--
> arch/x86/lib/csum-copy_64.S | 2
> arch/x86/lib/error-inject.c | 3 -
> arch/x86/lib/getuser.S | 22 +++++-----
> arch/x86/lib/hweight.S | 6 +-
> arch/x86/lib/iomap_copy_64.S | 2
> arch/x86/lib/memcpy_64.S | 12 ++---
> arch/x86/lib/memmove_64.S | 4 -
> arch/x86/lib/memset_64.S | 6 +-
> arch/x86/lib/msr-reg.S | 4 -
> arch/x86/lib/putuser.S | 6 +-
> arch/x86/lib/retpoline.S | 6 +-
> arch/x86/math-emu/div_Xsig.S | 2
> arch/x86/math-emu/div_small.S | 2
> arch/x86/math-emu/mul_Xsig.S | 6 +-
> arch/x86/math-emu/polynom_Xsig.S | 2
> arch/x86/math-emu/reg_norm.S | 6 +-
> arch/x86/math-emu/reg_round.S | 2
> arch/x86/math-emu/reg_u_add.S | 2
> arch/x86/math-emu/reg_u_div.S | 2
> arch/x86/math-emu/reg_u_mul.S | 2
> arch/x86/math-emu/reg_u_sub.S | 2
> arch/x86/math-emu/round_Xsig.S | 4 -
> arch/x86/math-emu/shr_Xsig.S | 8 +--
> arch/x86/math-emu/wm_shrx.S | 16 +++----
> arch/x86/mm/mem_encrypt_boot.S | 4 -
> arch/x86/platform/efi/efi_stub_32.S | 2
> arch/x86/platform/efi/efi_stub_64.S | 2
> arch/x86/platform/efi/efi_thunk_64.S | 2
> arch/x86/platform/olpc/xo1-wakeup.S | 6 +-
> arch/x86/power/hibernate_asm_32.S | 4 -
> arch/x86/power/hibernate_asm_64.S | 6 +-
> arch/x86/um/checksum_32.S | 4 -
> arch/x86/um/setjmp_32.S | 2
> arch/x86/um/setjmp_64.S | 2
> arch/x86/xen/xen-asm.S | 12 ++---
> arch/x86/xen/xen-head.S | 2
> samples/ftrace/ftrace-direct-modify.c | 4 -
> samples/ftrace/ftrace-direct-too.c | 2
> samples/ftrace/ftrace-direct.c | 2
> scripts/Makefile.lib | 3 -
> tools/objtool/arch/x86/decode.c | 13 ++++--
> tools/objtool/builtin-check.c | 3 -
> tools/objtool/check.c | 14 ++++++
> tools/objtool/include/objtool/arch.h | 1
> tools/objtool/include/objtool/builtin.h | 2
> 127 files changed, 447 insertions(+), 387 deletions(-)

It would be remiss of me not to say...  In Xen, I've currently gone with:

#ifdef CONFIG_SPECULATIVE_HARDEN_STRAIGHT_LINE
# define ret ret; int3
# define retq retq; int3
#endif


which avoids needing to patch any asm files, and therefore is a far less
invasive patch.

I know those defines probably deserve a WTF, and will confuse anyone who
doesn't understand the phrase "painting blue", but they have the
advantage that casual contributions don't need to remember to avoid
naked `ret` instructions to maintain safety, making it a rather more
robust option.

~Andrew