Re: [PATCH] crypto: rsa: add debug message if leading zero byte is missing
From: Kepplinger-Novakovic Martin
Date: Wed Feb 18 2026 - 04:23:01 EST
Am Mittwoch, dem 18.02.2026 um 10:06 +0100 schrieb Ignat Korchagin:
> On Wed, Feb 18, 2026 at 9:36 AM Kepplinger-Novakovic Martin
> <Martin.Kepplinger-Novakovic@xxxxxxxxxxxxx> wrote:
> >
> > Am Donnerstag, dem 12.02.2026 um 11:15 +0000 schrieb Ignat
> > Korchagin:
> > > Hi,
> > >
> > > On Thu, Feb 12, 2026 at 10:39 AM Martin Kepplinger-Novakovic
> > > <martin.kepplinger-novakovic@xxxxxxxxxxxxx> wrote:
> > > >
> > > > When debugging RSA certificate validation it can be valuable to
> > > > see
> > > > why the RSA verify() callback returns -EINVAL.
> > >
> > > Not sure if this would be some kind of an information leak
> > > (depending
> > > on a subsystem using RSA). Also what makes this case so special?
> > > Should we then annotate every other validation check in the code?
> > >
> > > > Signed-off-by: Martin Kepplinger-Novakovic
> > > > <martin.kepplinger-novakovic@xxxxxxxxxxxxx>
> > > > ---
> > > >
> > > > hi,
> > > >
> > > > my real issue is: When using a certificate based on an RSA-key,
> > > > I sometimes see signature-verify errors and (via dm-verity)
> > > > rootfs signature-verify errors, all triggered by "no leading 0
> > > > byte".
> > > >
> > > > key/cert generation:
> > > > openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out
> > > > ca.pem -
> > > > nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com
> > > > and simply used as trusted built-in key and rootfs hash sign
> > > > appended
> > > > to rootfs (squashfs).
> > > >
> > > > I'm on imx6ul. The thing is: Using the same certificate/key,
> > > > works
> > > > on
> > > > old v5.4-based kernels, up to v6.6!
> > > >
> > > > Starting with commit 2f1f34c1bf7b309 ("crypto: ahash - optimize
> > > > performance
> > > > when wrapping shash") it starts to break. it is not a commit on
> > > > it's own I
> > > > can revert and move on.
> > > >
> > > > What happended since v6.6 ? On v6.7 I see
> > > > [ 2.978722] caam_jr 2142000.jr: 40000013: DECO: desc idx 0:
> > > > Header Error. Invalid length or parity, or certain other
> > > > problems.
> > > >
> > > > and later the above -EINVAL from the RSA verify callback, where
> > > > I
> > > > add
> > > > the debug printing I see.
> > > >
> > > > What's the deal with this "leading 0 byte"?
> > >
> > > See RFC 2313, p 8.1
> >
> > hi Ignat,
> >
> > thanks for your time, the problem is *sometimes* rsa verify fails.
> > there seems to be a race condition:
>
> Can you clarify the failure case a bit? Is this the same signature
> that fails? (That is, you just verify a fixed signature in a loop) Or
> are these different signatures? (some reliably verify and some
> reliably fail)
>
different signuatures but nothing special: I add ca.pem (output of
"openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out ca.pem -
nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com") to
CONFIG_SYSTEM_TRUSTED_KEYS
during boot, asymmetric_key_preparse() is called, first on this, and
after that, "cfg80211: Loading compiled-in X.509 certificates for
regulatory database" does the same thing for Chen-Yu, Seth's keys in
mainline net/wireless/certs where I also added Ben's Debian
certificate.
The above verifications of 5 keys fail randomly.
In the end I (want to) use my own cert for dm-verity rootfs loading
(which also randomly fails).
on old kernels, most likely up to v6.6, there was no problem.
thank you for asking,
martin
> > in the failure-case after crypto_akcipher_encrypt() and
> > crypto_wait_req() the *same* data as before is still at out_buf!
> > that
> > has not yet been written to.
> >
> > It's not that obvious to be yet because msleep(1000) doesn't change
> > much and 00, 01, ff, ff... is *still* not yet written to out_buf!
> >
> > is there a reason why crypto_akcipher_sync_{en,de}crypt() is not
> > used?
> > Can you imagine what could go wrong here?
> >
> > *maybe* commit 1e562deacecca1f1bec7d23da526904a1e87525e that did a
> > lot
> > of things in parallel (in order to keep functionality similar) got
> > something wrong?
> >
> > sidenote: when I use an elliptic curve key instead of rsa,
> > everything
> > works.
> >
> > also, the auto-free for child_req looks a bit dangerous when using
> > out_buf, but ok :)
> >
> > maybe this rings a bell, I'll keep debugging,
> >
> > martin
> >
> >
> > >
> > > >
> > > > thank you!
> > > >
> > > > martin
> > > >
> > > >
> > > >
> > > > crypto/rsa-pkcs1pad.c | 5 +++--
> > > > crypto/rsassa-pkcs1.c | 5 +++--
> > > > 2 files changed, 6 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/crypto/rsa-pkcs1pad.c b/crypto/rsa-pkcs1pad.c
> > > > index 50bdb18e7b483..65a4821e9758b 100644
> > > > --- a/crypto/rsa-pkcs1pad.c
> > > > +++ b/crypto/rsa-pkcs1pad.c
> > > > @@ -191,9 +191,10 @@ static int
> > > > pkcs1pad_decrypt_complete(struct
> > > > akcipher_request *req, int err)
> > > >
> > > > out_buf = req_ctx->out_buf;
> > > > if (dst_len == ctx->key_size) {
> > > > - if (out_buf[0] != 0x00)
> > > > - /* Decrypted value had no leading 0
> > > > byte */
> > > > + if (out_buf[0] != 0x00) {
> > > > + pr_debug("Decrypted value had no
> > > > leading 0
> > > > byte\n");
> > > > goto done;
> > > > + }
> > > >
> > > > dst_len--;
> > > > out_buf++;
> > > > diff --git a/crypto/rsassa-pkcs1.c b/crypto/rsassa-pkcs1.c
> > > > index 94fa5e9600e79..22919728ea1c8 100644
> > > > --- a/crypto/rsassa-pkcs1.c
> > > > +++ b/crypto/rsassa-pkcs1.c
> > > > @@ -263,9 +263,10 @@ static int rsassa_pkcs1_verify(struct
> > > > crypto_sig *tfm,
> > > > return -EINVAL;
> > > >
> > > > if (dst_len == ctx->key_size) {
> > > > - if (out_buf[0] != 0x00)
> > > > - /* Encrypted value had no leading 0
> > > > byte */
> > > > + if (out_buf[0] != 0x00) {
> > > > + pr_debug("Encrypted value had no
> > > > leading 0
> > > > byte\n");
> > > > return -EINVAL;
> > > > + }
> > > >
> > > > dst_len--;
> > > > out_buf++;
> > > > --
> > > > 2.47.3
> > > >
> > >
> > > Ignat