Re: [tip: x86/asm] x86/cpufeatures: Add support for fast short REP; MOVSB

[Ingo Molnar]

* tip-bot2 for Tony Luck <tip-bot2@xxxxxxxxxxxxx> wrote:

> The following commit has been merged into the x86/asm branch of tip:
> 
> Commit-ID:     f444a5ff95dce07cf4353cbb85fc3e785019d430
> Gitweb:        https://git.kernel.org/tip/f444a5ff95dce07cf4353cbb85fc3e785019d430
> Author:        Tony Luck <tony.luck@xxxxxxxxx>
> AuthorDate:    Mon, 16 Dec 2019 13:42:54 -08:00
> Committer:     Borislav Petkov <bp@xxxxxxx>
> CommitterDate: Wed, 08 Jan 2020 11:29:25 +01:00
> 
> x86/cpufeatures: Add support for fast short REP; MOVSB
> 
> >From the Intel Optimization Reference Manual:
> 
> 3.7.6.1 Fast Short REP MOVSB
> Beginning with processors based on Ice Lake Client microarchitecture,
> REP MOVSB performance of short operations is enhanced. The enhancement
> applies to string lengths between 1 and 128 bytes long.  Support for
> fast-short REP MOVSB is enumerated by the CPUID feature flag: CPUID
> [EAX=7H, ECX=0H).EDX.FAST_SHORT_REP_MOVSB[bit 4] = 1. There is no change
> in the REP STOS performance.
> 
> Add an X86_FEATURE_FSRM flag for this.
> 
> memmove() avoids REP MOVSB for short (< 32 byte) copies. Check FSRM and
> use REP MOVSB for short copies on systems that support it.
> 
>  [ bp: Massage and add comment. ]
> 
> Signed-off-by: Tony Luck <tony.luck@xxxxxxxxx>
> Signed-off-by: Borislav Petkov <bp@xxxxxxx>
> Link: https://lkml.kernel.org/r/20191216214254.26492-1-tony.luck@xxxxxxxxx

BTW., just for the record, the 32-bit version of memmove() has a similar 
cut-off as well, at 680 bytes (!):

                /*
                 * movs instruction have many startup latency
                 * so we handle small size by general register.
                 */
                "cmp  $680, %0\n\t"
                "jb 3f\n\t"

...

                /*
                 * Start to prepare for backward copy.
                 */
                ".p2align 4\n\t"
                "2:\n\t"
                "cmp  $680, %0\n\t"
                "jb 5f\n\t"

This logic was introduced in 2010 via:

   3b4b682becdf: ("x86, mem: Optimize memmove for small size and unaligned cases")

However because those patches came without actual performance 
measurements, I'd be inclined to switch back to the old REP MOVSB version 
- which would also automatically improve it should anyone run 32-bit 
kernels on the very latest CPUs.

Thanks,

	Ingo