Re: [PATCH] lib/crc: Add 4-way folding in NEON crc64_nvme implementation

From: Ard Biesheuvel

Date: Wed Nov 18 2026 - 11:30:11 EST



On Wed, 18 Nov 2026, at 17:01, Shreesh Adiga wrote:
> Use 4-way folding in the shared NEON crc64_nvme for AArch64
> as well as AArch32. This improves performance by about 2x for 512 byte
> data size on Cortex-X925.
>
> Add similar optimizations from x86 implementation like reducing
> the 4x to 2x and doing a tail (len >= 32) 2x fold as well as partial
> block handling to process lengths which are not multiple of 16 bytes.
>
> The partial block stage is enabled only for AArch64 as it needs the
> 128-bit TBL instruction not available in AArch32.
> Modified the loop dispatch on AArch64 to allow any len >= 128 rather
> than just multiple of 16 bytes.
>
> Modified the pmull intrinsics from crc64-neon.h to use inline
> assembly as the intrinsic usage was causing the compiler to
> generate the fold constants via mov+movk+fmov/dup which was about 10
> instructions for 16-byte constant instead of expected adrp+ld1.
>

Why is that better?