Re: [PATCH v3 2/2] x86/lib: Use REP MOVSB for large memcpy() on ERMS CPUs without FSRM
From: Borislav Petkov
Date: Wed Nov 11 2026 - 18:59:21 EST
On Wed, Nov 11, 2026 at 05:20:53AM -0800, Usama Arif wrote:
> diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S
> index 7bb146c811029..5320862e0274e 100644
> --- a/arch/x86/lib/memcpy_64.S
> +++ b/arch/x86/lib/memcpy_64.S
> @@ -27,15 +27,13 @@
> *
> * The 'rep movsb' itself is small enough to replace the call, but the
> * two register moves blow up the code. And one of them is "needed"
> - * only for the return value that is the same as the source input,
> + * only for the return value that is the same as the destination input,
> * which the compiler could/should do much better anyway.
> */
> SYM_TYPED_FUNC_START(__memcpy)
> - ALTERNATIVE "jmp memcpy_orig", "", X86_FEATURE_FSRM
> -
> movq %rdi, %rax
> movq %rdx, %rcx
> - rep movsb
> + ALTERNATIVE "jmp memcpy_orig", "rep movsb", X86_FEATURE_FSRM
> RET
> SYM_FUNC_END(__memcpy)
> EXPORT_SYMBOL(__memcpy)
> @@ -45,12 +43,16 @@ SYM_PIC_ALIAS(memcpy)
> EXPORT_SYMBOL(memcpy)
>
> SYM_FUNC_START_LOCAL(memcpy_orig)
> - movq %rdi, %rax
> - movq %rdx, %rcx
> -
> cmpq $0x20, %rcx
> jb .Lhandle_tail
>
> + /*
> + * RCX holds the count for REP MOVSB and scalar copying; RDX is scratch.
> + * Shift RDX to check length >= 2048 without destroying the REP count.
> + * The 10-byte alternative fits a NOP without ERMS.
> + */
> + ALTERNATIVE "", __stringify(shrq $11, %rdx; jnz .Lerms), X86_FEATURE_ERMS
Sorry, this is adding more JMPs in "conditional" alternatives code and thus
turns this into a bigger and unmaintainable mess than it already is.
The 2K limit looks like something conjured out of a thin air as it seems it
work for your particular case. I'm sure we will have to whack it in the future
because someone else will complain of the opposite.
This needs to be done with alternative_call*, even at the price of some
duplicated code but without turning what already is a mess, into a bigger one.
Or a static key or even with stop_machine_nmi() (currently in review) which
patches the kernel even later, when it is convenient to do so.
Whatever it takes to keep this code readable and sane.
But not this.
Sorry.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette