Re: [PATCH v2] lib/crc: arm64: Add NEON PMULL+EOR3 crc64_nvme implementation

From: Eric Biggers

Date: Tue Nov 17 2026 - 02:12:03 EST


On Tue, Nov 17, 2026 at 12:20:08PM +0530, Shreesh Adiga wrote:
> Hi Eric,
>
> On Tue, Nov 17, 2026 at 5:31 AM Eric Biggers <ebiggers@xxxxxxxxxx> wrote:
> > Was crc_kunit run? It already has a benchmark in it.
> No, I have access to only one aarch64 device that is Android Mediatek
> Dimensity 9400 SoC without root access.
> Therefore I can't run crc_kunit on that, instead I have done userspace
> comparisons for the same kernel with libbenchmark.
> This is based on my earlier work for FFmpeg:
> https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21651
> I also have Corporate MacOS M5 and did check inside Ubuntu VM but that
> is not accurate due to virtualization.

Well, this is a kernel patch. Contributors are expected to test their
kernel patches. It can at least be done with QEMU.

> > The commit message should also clarify that this benefits ARMv8.2 and
> > later processors with the sha3 extension, not just ARMv9. (E.g., Apple
> > M1 is an example of an ARMv8.4 processor that benefits.)
> Sure I will take care of it, but for now I am thinking of first
> improving the existing PMULL to have 4x unrolling as per Demian's
> suggestion.

That would be fine too.

> > Note that this is missing several optimizations compared to the x86
> > version (lib/crc/x86/crc-pclmul-template.S), including using
> > fold_consts_2x for a tail of 'len & 32', efficiently handling lengths
> > that aren't a multiple of 16 bytes, and reducing to the final CRC in two
> > carryless multiplication instructions rather than three.
> I was looking at the crc-pclmul-template.S code and I have a few queries.
> In this aarch64 version we only dispatch to this SIMD routine only
> when len>=128.
> Since we have 128 bytes available, do we have to add the tail of `len
> & 32` handling?

It's still useful, since the length mod 64 might be >= 32.

> Handling len not a multiple of 16 bytes (partial block) I will add but
> I am assuming that the dispatch will maintain 128 minimum len.

There is overhead in entering a kernel-mode NEON section, so it makes
sense to continue to have a minimum length.

> I was trying to understand your "reducing to the final CRC in two
> carryless multiplications rather than three" statement.
> From going through the code in crc-pclmul-template.S:
> .Lreduce_128bits_to_crc tracing the logic below this label, I see
> three clmul operations:
> _pclmulqdq CONSTS_XMM, LO64_TERMS, %xmm0, HI64_TERMS, %xmm1
>
> _cond_vex movdqa, OFFSETOF_BARRETT_REDUCTION_CONSTS(CONSTS_PTR), CONSTS_XMM
> _pclmulqdq CONSTS_XMM, HI64_TERMS, %xmm0, HI64_TERMS, %xmm1
>
> .if LSB_CRC && \n == 64
> _cond_vex punpcklqdq, %xmm1, %xmm2, %xmm2
> _pclmulqdq CONSTS_XMM, LO64_TERMS, %xmm1, HI64_TERMS, %xmm1
>
> I even checked the Intel Whitepaper and the reduction for 64-bit CRC
> section "Final Reduction of 128-bits" also mentions 3 clmul.
> One for using x^128 followed by xor, and then 2 more for final barrett
> reduction with mu and poly.
> Thus I am confused about accomplishing this with just 2 clmul. Could
> you provide details on this?

Sorry, I misremembered. It's indeed 3 minimum. There are just a lot of
implementations out there that unnecessarily do 4, including
lib/crc/arm64/crc-t10dif-core.S for example which still exists.

- Eric