[PATCH] crypto: rsa: add debug message if leading zero byte is missing

[Martin Kepplinger-Novakovic]

When debugging RSA certificate validation it can be valuable to see
why the RSA verify() callback returns -EINVAL.

Signed-off-by: Martin Kepplinger-Novakovic <martin.kepplinger-novakovic@xxxxxxxxxxxxx>
---

hi,

my real issue is: When using a certificate based on an RSA-key,
I sometimes see signature-verify errors and (via dm-verity)
rootfs signature-verify errors, all triggered by "no leading 0 byte".

key/cert generation:
openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out ca.pem -nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com

and simply used as trusted built-in key and rootfs hash sign appended
to rootfs (squashfs).

I'm on imx6ul. The thing is: Using the same certificate/key, works on
old v5.4-based kernels, up to v6.6!

Starting with commit 2f1f34c1bf7b309 ("crypto: ahash - optimize performance
when wrapping shash") it starts to break. it is not a commit on it's own I
can revert and move on.

What happended since v6.6 ? On v6.7 I see
[    2.978722] caam_jr 2142000.jr: 40000013: DECO: desc idx 0: Header Error. Invalid length or parity, or certain other problems.

and later the above -EINVAL from the RSA verify callback, where I add
the debug printing I see.

What's the deal with this "leading 0 byte"?


thank you!

                                    martin



 crypto/rsa-pkcs1pad.c | 5 +++--
 crypto/rsassa-pkcs1.c | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/crypto/rsa-pkcs1pad.c b/crypto/rsa-pkcs1pad.c
index 50bdb18e7b483..65a4821e9758b 100644
--- a/crypto/rsa-pkcs1pad.c
+++ b/crypto/rsa-pkcs1pad.c
@@ -191,9 +191,10 @@ static int pkcs1pad_decrypt_complete(struct akcipher_request *req, int err)
 
 	out_buf = req_ctx->out_buf;
 	if (dst_len == ctx->key_size) {
-		if (out_buf[0] != 0x00)
-			/* Decrypted value had no leading 0 byte */
+		if (out_buf[0] != 0x00) {
+			pr_debug("Decrypted value had no leading 0 byte\n");
 			goto done;
+		}
 
 		dst_len--;
 		out_buf++;
diff --git a/crypto/rsassa-pkcs1.c b/crypto/rsassa-pkcs1.c
index 94fa5e9600e79..22919728ea1c8 100644
--- a/crypto/rsassa-pkcs1.c
+++ b/crypto/rsassa-pkcs1.c
@@ -263,9 +263,10 @@ static int rsassa_pkcs1_verify(struct crypto_sig *tfm,
 		return -EINVAL;
 
 	if (dst_len == ctx->key_size) {
-		if (out_buf[0] != 0x00)
-			/* Encrypted value had no leading 0 byte */
+		if (out_buf[0] != 0x00) {
+			pr_debug("Encrypted value had no leading 0 byte\n");
 			return -EINVAL;
+		}
 
 		dst_len--;
 		out_buf++;
-- 
2.47.3