Re: [BUG] crypto: caam - RSA encrypt doesn't always complete new data in out_buf

[Lukas Wunner]

On Tue, Feb 24, 2026 at 02:17:22PM +0000, Kepplinger-Novakovic Martin wrote:
> I run imx6ul with FSL_CAAM* enabled and simply add ca.pem 
> ( openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out ca.pem \
> -nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com )
> to CONFIG_SYSTEM_TRUSTED_KEYS in order to use it to verify a squashfs
> rootfs via dm-verity (but forget that for a moment, the failure is early
> during boot, rsa verify()).

The issue might be easier to debug if you could come up with a
simpler reproducer.  E.g. you could use keyctl(1) to add a key to
a keyring in the kernel and subsequently have the kernel verify
a signature with it.

> This works until v6.6 and fails after ("crypto: ahash - optimize
> performance when wrapping shash")
> but too much has happened that I could revert one and I might be wrong
> with that commit even.

It would be good if you could bisect to exactly pinpoint the offending
commit.

> I run v6.18 now where this works when I *disable* FSL_CAAM* completely
> and use the rsa-generic code.
> Also it works, when I generate from elliptic curve key.
> 
> Only the CAAM+RSA case is failing in a weird way:
[...]
> but why? Because after the CAAM completion callback returning to the
> caller, out_buf (that was set to the key-part inside of the request
> earlier) still holds *old* data,
> and NOT the "encyption block" with 00 01 ff ff.... 
> 
> out_buf1 is (I assume valid because never changing) input data "out_buf"
> inside of "child_req".

"git grep out_buf1" finds nothing in a v6.19 source tree, so I'm not sure
what you're referring to?

> Directly when caam jr dequeue() returns with the user-callback, I see 64 bytes (all?)
> old data. SOMEHOW the cryto-wait needs 100ms or longer (no idea why) and after THAT,
> out_buf has only the first 16 bytes old data (out_buf2 in the logs):

I assume the caam device writes to memory via DMA.  Perhaps the CPU isn't
aware that the memory has been changed and is using data in its cache?
That could be caused by an incorrect dma_map_*() call in caampkc.c.

> There has been quite some changes, most notably ("crypto: ahash - optimize
> performance when wrapping shash")
> or ("crypto: rsassa-pkcs1 - Migrate to sig_alg backend") which squashes
> differnt changes into one commit...

Being the author of the latter, your initial report scared me that
I might have broken something.  However I looked through the code
and nothing stuck out.

Thanks,

Lukas