Re: [PATCH v3 0/2] x86/lib: Use REP MOVSB for large memcpy() on ERMS CPUs without FSRM

From: Kiryl Shutsemau

Date: Wed Nov 11 2026 - 13:31:37 EST


On Wed, Nov 11, 2026 at 05:20:51AM -0800, Usama Arif wrote:
> Enable REP MOVSB for copies of 2048 bytes and larger on ERMS CPUs without
> FSRM, keeping scalar copies below that threshold. FSRM CPUs continue to use
> REP MOVSB for every size.
>
> This series splits v2 into 2 patches.
> The first changes only the scalar register allocation: RCX holds the count
> and tail indexes, and RDX is the small-copy data temporary. The second
> adds the ERMS size check and adopts Linus's FSRM entry layout. REP MOVSB
> replaces the existing jump to memcpy_orig(), with RET remaining in
> ordinary text. ITS therefore selects return handling using the return's
> actual execution address, without aligning replacement storage. Keeping
> the count in RCX also addresses David's request to avoid reloading it on
> the ERMS path.
>
> v2 -> v3: https://lore.kernel.org/r/20261104222806.1826508-1-usama.arif@xxxxxxxxx
> - Split the scalar RCX/RDX register swap into a preparatory patch with no
> functional change (Kiryl Shutsemau).
> - Use REP MOVSB to replace the jump to memcpy_orig() on FSRM CPUs, keeping
> RET in ordinary text (Kiryl Shutsemau, Linus Torvalds). Remove the ITS
> replacement alignment, since no return is copied by alternatives now.
>
> v1 -> v2: https://lore.kernel.org/r/20261013100442.2658311-1-usama.arif@xxxxxxxxx
> - Raise the threshold from 512 to 2048 bytes. REP MOVSB was slower than
> the scalar loop for some misaligned copies of 512 to 640 bytes.
> - Fold memcpy_orig() into __memcpy(). With FSRM, "rep movsb; RET" now
> replaces the 32-byte length check (David Laight, Linus Torvalds). In
> v1, the 13-byte empty FSRM replacement became a jump that FSRM CPUs
> took on every call (Kiryl Shutsemau).
> - Do the ERMS check after the 32-byte check, so copies below 32 bytes
> skip it (Kiryl Shutsemau), and keep the count in RCX so REP MOVSB
> needs no reload (David Laight).
> - Align the FSRM replacement so that its RET keeps the ITS return thunk.
> - Fix the comment that called the return value the source input.
>
> Usama Arif (2):
> x86/lib: Keep the scalar memcpy() count in RCX
> x86/lib: Use REP MOVSB for large memcpy() on ERMS CPUs without FSRM
>
> arch/x86/lib/memcpy_64.S | 78 +++++++++++++++++++++-------------------
> 1 file changed, 42 insertions(+), 36 deletions(-)

Acked-by: Kiryl Shutsemau (Meta) <kas@xxxxxxxxxx>

--
Kiryl Shutsemau / Kirill A. Shutemov