[PATCH] gpu: nova-core: gsp: zero-pad command queue elements
From: haider
Date: Sun Oct 11 2026 - 12:16:46 EST
GSP-RM checksums a command queue element by XORing it in 64-bit words,
so for an element whose size is not a multiple of 8 it also reads up to
7 bytes past its end, and expects them to be zero: its _checkSum32()
"assumes that the data is padded out with zeros to the next 8-byte
alignment". NVIDIA's kernel driver zeroes these bytes before computing
the checksum, and nouveau builds each element in a zeroed buffer.
allocate_command() hands out the queue slots as they are, so these bytes
still hold whatever an earlier, longer element left there. Once the
queue has wrapped, a command of such a size can then fail GSP-RM's
checksum: GSP-RM never answers it, and soon stops answering at all.
Nova does not send enough commands for the queue to wrap yet, as it is
zeroed at probe, but this will break as soon as it does.
Clear each element and its padding before writing it.
Fixes: 75f6b1de8133 ("gpu: nova-core: gsp: Add GSP command queue bindings and handling")
Assisted-by: LLM
Signed-off-by: haider <soka11223344k@xxxxxxxxx>
---
Notes:
Found and tested on an out-of-tree nova branch that sends RM object
allocations and controls through the command queue, so the queue wraps
there. Once it had wrapped, the allocation of a subdevice failed the
checksum, and GSP-RM stopped answering, logging Display NOCAT 0x28,
assertions and Xid 45, within the first 5-10 cycles of creating and
freeing a client, device and subdevice. With this patch, 288 such cycles
ran clean (RTX 3070 / GA104, GSP firmware 570.144).
On drm-rust-next as is the queue never wraps, so there it is only
build-tested (W=1, CLIPPY=1), along with checkpatch and rustfmt.
An LLM found the cause by comparing the command queue code with NVIDIA's
open kernel modules and nouveau, and wrote the fix. I reviewed and
tested it.
drivers/gpu/nova-core/gsp/cmdq.rs | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 1b2f346a23..ff323bb37c 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -384,6 +384,15 @@ fn allocate_command(&mut self, size: usize, timeout: Delta) -> Result<GspCommand
(slice_1.as_flattened_mut(), slice_2.as_flattened_mut())
};
+ // GSP-RM checksums an element in 64-bit words, reading up to 7 bytes past its end, and
+ // expects them to be zero (`_checkSum32` in NVIDIA's `message_queue_priv.h`). The ring
+ // still holds what earlier elements left there, so clear the element and its padding.
+ // Elements take whole pages, so the padding fits in the space that was waited for.
+ let padded = (size_of::<GspMsgElement>() + size).next_multiple_of(size_of::<u64>());
+ let first = padded.min(slice_1.len());
+ slice_1[..first].fill(0);
+ slice_2.get_mut(..padded - first).ok_or(EIO)?.fill(0);
+
// Extract area for the `GspMsgElement`.
let (header, slice_1) = GspMsgElement::from_bytes_mut_prefix(slice_1).ok_or(EIO)?;
base-commit: 5a1b7121a6c4db6c0239d3ac3fd1c2bb33a40025
--
2.55.0