Re: [PATCH v4 3/3] arm64/crc32: Implement 4-way interleave using PMULL
From: Eric Biggers
Date: Fri Oct 18 2024 - 19:56:24 EST
On Fri, Oct 18, 2024 at 09:53:51AM +0200, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <ardb@xxxxxxxxxx>
>
> Now that kernel mode NEON no longer disables preemption, using FP/SIMD
> in library code which is not obviously part of the crypto subsystem is
> no longer problematic, as it will no longer incur unexpected latencies.
>
> So accelerate the CRC-32 library code on arm64 to use a 4-way
> interleave, using PMULL instructions to implement the folding.
>
> On Apple M2, this results in a speedup of 2 - 2.8x when using input
> sizes of 1k - 8k. For smaller sizes, the overhead of preserving and
> restoring the FP/SIMD register file may not be worth it, so 1k is used
> as a threshold for choosing this code path.
>
> The coefficient tables were generated using code provided by Eric. [0]
>
> [0] https://github.com/ebiggers/libdeflate/blob/master/scripts/gen_crc32_multipliers.c
>
> Cc: Eric Biggers <ebiggers@xxxxxxxxxx>
> Signed-off-by: Ard Biesheuvel <ardb@xxxxxxxxxx>
> ---
> arch/arm64/lib/crc32-glue.c | 48 ++++
> arch/arm64/lib/crc32.S | 231 +++++++++++++++++++-
> 2 files changed, 276 insertions(+), 3 deletions(-)
Reviewed-by: Eric Biggers <ebiggers@xxxxxxxxxx>
> + if (len >= min_len && cpu_have_named_feature(PMULL) && crypto_simd_usable()) {
Using crypto_simd_usable() here causes a build error when CRYPTO_ALGAPI2=m.
https://lore.kernel.org/linux-crypto/20241018235343.425758-1-ebiggers@xxxxxxxxxx
will fix that.
- Eric