[PATCH v2 00/19] crypto: x86 - fix RCU stalls

[Robert Elliott]

This series fixes the RCU stalls triggered by the x86 crypto
modules discussed in
https://lore.kernel.org/all/MW5PR84MB18426EBBA3303770A8BC0BDFAB759@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/

Two root causes were:
- too much data processed between kernel_fpu_begin and
  kernel_fpu_end calls (which are heavily used by the x86
  optimized drivers)
- tcrypt not calling cond_resched during speed test loops

These problems have always been lurking, but improving the
loading of the x86/sha512 module led to it happening a lot
during boot when using SHA-512 for module signature checking. 

Fixing these problems makes it safer to improve loading
the rest of the x86 modules like the sha512 module.

This series only handles the x86 modules.

Testing
=======
The most effective testing was by enabling
  CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y

which creates random test vectors and compares the results
of the CPU-optimized function to the generic function,
and running two threads of repeated modprobe commands
to exercise those tests:
  watch -n 0 modprobe tcrypt mode=200
  watch -n 0 ./tcrypt_sweep

where tcrypt_sweep walks through all the test modes:
#!/usr/bin/perl
use strict;

my @modes;

open SOURCE, "<", "/home/me/linux/crypto/tcrypt.c" or die $!;
while (<SOURCE>) {
        if (/^\s+case ([0-9]+):$/) {
                push @modes, $1;
        }
}
close SOURCE;

foreach (@modes) {
        print "$_ ";

        # don't run mode 300, which runs 301-399
        # don't run mode 400, which runs 401-499
        if (($_ eq "0") || ($_ eq "300") || ($_ eq "400")) {
                system "echo \"===== Skipping special modprobe tcrypt mode=$_\" > /dev/kmsg";
        } else {
                system "echo \"Running modprobe tcrypt mode=$_\" > /dev/kmsg";
                system "modprobe tcrypt mode=$_";
        }
}



Robert Elliott (19):
  crypto: tcrypt - test crc32
  crypto: tcrypt - test nhpoly1305
  crypto: tcrypt - reschedule during cycles speed tests
  crypto: x86/sha - limit FPU preemption
  crypto: x86/crc - limit FPU preemption
  crypto: x86/sm3 - limit FPU preemption
  crypto: x86/ghash - restructure FPU context saving
  crypto: x86/ghash - limit FPU preemption
  crypto: x86 - use common macro for FPU limit
  crypto: x86/sha1, sha256 - load based on CPU features
  crypto: x86/crc - load based on CPU features
  crypto: x86/sm3 - load based on CPU features
  crypto: x86/ghash - load based on CPU features
  crypto: x86 - load based on CPU features
  crypto: x86 - add pr_fmt to all modules
  crypto: x86 - print CPU optimized loaded messages
  crypto: x86 - standardize suboptimal prints
  crypto: x86 - standardize not loaded prints
  crypto: x86/sha - register only the best function

 arch/x86/crypto/aegis128-aesni-glue.c      |  21 ++-
 arch/x86/crypto/aesni-intel_glue.c         |  31 ++--
 arch/x86/crypto/aria_aesni_avx_glue.c      |  19 +-
 arch/x86/crypto/blake2s-glue.c             |  34 +++-
 arch/x86/crypto/blowfish_glue.c            |  19 +-
 arch/x86/crypto/camellia_aesni_avx2_glue.c |  25 ++-
 arch/x86/crypto/camellia_aesni_avx_glue.c  |  24 ++-
 arch/x86/crypto/camellia_glue.c            |  20 ++-
 arch/x86/crypto/cast5_avx_glue.c           |  21 ++-
 arch/x86/crypto/cast6_avx_glue.c           |  21 ++-
 arch/x86/crypto/chacha_glue.c              |  35 +++-
 arch/x86/crypto/crc32-pclmul_asm.S         |   6 +-
 arch/x86/crypto/crc32-pclmul_glue.c        |  37 ++--
 arch/x86/crypto/crc32c-intel_glue.c        |  51 ++++--
 arch/x86/crypto/crct10dif-pclmul_glue.c    |  54 ++++--
 arch/x86/crypto/curve25519-x86_64.c        |  27 ++-
 arch/x86/crypto/des3_ede_glue.c            |  16 +-
 arch/x86/crypto/ghash-clmulni-intel_glue.c |  40 +++--
 arch/x86/crypto/nhpoly1305-avx2-glue.c     |  27 ++-
 arch/x86/crypto/nhpoly1305-sse2-glue.c     |  23 ++-
 arch/x86/crypto/poly1305_glue.c            |  64 +++++--
 arch/x86/crypto/polyval-clmulni_glue.c     |  14 +-
 arch/x86/crypto/serpent_avx2_glue.c        |  25 ++-
 arch/x86/crypto/serpent_avx_glue.c         |  21 ++-
 arch/x86/crypto/serpent_sse2_glue.c        |  19 +-
 arch/x86/crypto/sha1_ssse3_glue.c          | 188 +++++++++++--------
 arch/x86/crypto/sha256_ssse3_glue.c        | 198 ++++++++++++---------
 arch/x86/crypto/sha512_ssse3_glue.c        | 154 +++++++++-------
 arch/x86/crypto/sm3_avx_glue.c             |  52 +++++-
 arch/x86/crypto/sm4_aesni_avx2_glue.c      |  25 ++-
 arch/x86/crypto/sm4_aesni_avx_glue.c       |  23 ++-
 arch/x86/crypto/twofish_avx_glue.c         |  25 ++-
 arch/x86/crypto/twofish_glue.c             |  19 +-
 arch/x86/crypto/twofish_glue_3way.c        |  26 ++-
 crypto/tcrypt.c                            |  56 +++---
 35 files changed, 1060 insertions(+), 400 deletions(-)

-- 
2.37.3