Re: [PATCH] x86/lib: Use REP MOVSB for large memcpy() on ERMS CPUs without FSRM

[Usama Arif]

On Thu, 15 Oct 2026 11:39:10 +0100 Kiryl Shutsemau <kas@xxxxxxxxxx> wrote:

> On Tue, Oct 13, 2026 at 03:04:42AM -0700, Usama Arif wrote:
> >  SYM_TYPED_FUNC_START(__memcpy)
> > -	ALTERNATIVE "jmp memcpy_orig", "", X86_FEATURE_FSRM
> > +	ALTERNATIVE_2 "jmp memcpy_orig", \
> > +		      "cmpq $512, %rdx; jb memcpy_orig", X86_FEATURE_ERMS, \
> > +		      "", X86_FEATURE_FSRM
> 
> This grows the patch site from 2 bytes to 13. On FSRM the replacement is
> empty, and 13 bytes is more than ASM_NOP_MAX, so add_nop() puts an
> unconditional jump here, executed on every memcpy() call on FSRM
> machines. :/
> 

Thanks for the review and catching the FSRM padding jump! You're right,
the 13-byte empty replacement introduces an unconditional jump on every call.

> Maybe it is worth moving the patch site into memcpy_orig()? It already
> has a 32-byte check. Which brings me to the other question:

I tested the 32-byte cutoff on Cooper Lake, which has ERMS without FSRM.
With warmed, aligned buffers, 32-byte copies took 10.906 ns with the
parent and 23.148 ns with the proposal (+112%). The increases at 64, 128 and
256 bytes were about 66%, 65% and 22%. These regressions reproduced across
five process runs and all four tested entry placements.
 
I found 2K to be the spot where REP MOVSB is better. For the entry, I moved
the destination save into the FSRM alternative:

ALTERNATIVE_2 "jmp memcpy_orig", "jmp memcpy_erms", X86_FEATURE_ERMS, \
              "movq %rdi, %rax", X86_FEATURE_FSRM

The ordinary entry then only needs MOV to RCX, REP MOVSB and RET. This gives
FSRM a destination move plus one two-byte NOP, without a padding jump, and
keeps the entry size and scalar layout. A separate memcpy_erms helper checks
2048 and falls back to memcpy_orig below that size. Returns stay in normal
text.

Below is the current state of my patch. What do you think?