Re: [PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly
From: Detlev Casanova
Date: Tue May 19 2026 - 09:06:09 EST
Hi Michael,
On 5/13/26 14:19, Michael Bommarito wrote:
V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS andStill, did you try just changing the cap to 64 (.cfg.dims = { 64 },) ?
V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS are registered as dynamic-size
controls with a per-control element cap of 65. The V4L2 control core
enforces only the payload-element cap. It does not bound the spec-derived
count fields that the rkvdec HEVC helper later uses to walk fixed hardware
descriptor tables and temporary helper arrays:
- struct rkvdec_rps::refs[32]
- struct rkvdec_rps::short_term_ref_sets[64]
- struct calculated_rps_st_set::delta_poc_s0[16] / delta_poc_s1[16]
You'd need a test that sets the control from userspace though.
It should refuse setting the control if there are more than 64 elements, therefore the hevc decoder will not run any function using the count values from the SPS (See rkvdec-vdpu381-hevc.c:601)
A userspace V4L2 client that can open the Rockchip RKVDEC m2m decoder node
may submit SPS/RPS controls whose counts exceed those capacities or whose
prediction reference index underflows. rkvdec_hevc_assemble_hw_rps() then
walks past the descriptor table or temporary-array bounds.
KASAN under a small KUnit harness wrapping the real helper reports
slab-out-of-bounds in all of:
- num_short_term_ref_pic_sets = 65 (write past short_term_ref_sets[64])
- num_long_term_ref_pics_sps = 33 (write past refs[32], intra-struct)
- ext_sps_st_rps[i].num_negative_pics or num_positive_pics > 16
(write past delta_poc_s0[16] inside calculated_rps_st_set)
- INTER_REF_PIC_SET_PRED with delta_idx_minus1 + 1 > i
(u8 ref_rps_idx underflow then OOB read on calculated_rps_st_sets)
Validate the SPS/RPS counts before calling the assembly helpers. The cap
values match both the HEVC spec ranges (num_short_term_ref_pic_sets <= 64,
num_long_term_ref_pics_sps <= 32) and the fixed driver descriptor and
helper-array capacities. Reject controls whose counts exceed those, and
reject prediction entries whose reference index would underflow.
Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
Signed-off-by: Michael Bommarito <michael.bommarito@xxxxxxxxx>
Assisted-by: Claude:claude-opus-4-7
---
I don't have a RK3588 / RK3576 board to confirm this through a real
/dev/videoN request path yet, but was convinced enough by:
1. Static reach: registration of EXT_SPS_ST_RPS / EXT_SPS_LT_RPS with
.dims = { 65 } at drivers/media/platform/rockchip/rkvdec/rkvdec.c
:239-287, the SPS-count-driven loops in rkvdec-hevc-common.c
:213-225 and :228-251, and v4l2-ctrls-core.c :1213-1277, which
validates only EXT RPS flags and not the spec-derived count fields.
2. A KUnit harness (separate, not in this patch) that allocates one
struct rkvdec_rps + a single calculated_rps_st_sets element via
kunit_kzalloc / kzalloc and calls the real
rkvdec_hevc_assemble_hw_rps() helper. Under UML + KASAN_GENERIC with
the kasan_multi_shot boot param, on a stock tree it produces these
reports:
BUG: KASAN: slab-out-of-bounds in
rkvdec_hevc_assemble_hw_rps+0xb0c/0x1080
(num_short_term_ref_pic_sets = 65, write of size 36 0 bytes past
the rps allocation)
BUG: KASAN: slab-out-of-bounds in
rkvdec_hevc_assemble_hw_rps (num_negative_pics = 64, write past
the single-element kzalloc'd calculated_rps_st_sets buffer)
BUG: KASAN: slab-use-after-free / slab-out-of-bounds reads via
u8 ref_rps_idx underflow at calculated_rps_st_sets[255]
(INTER_REF_PIC_SET_PRED with delta_idx_minus1 = 0, idx = 0)
The num_long_term_ref_pics_sps = 33 case is invisible to KASAN
(the OOB write lands inside struct rkvdec_rps) but corrupts
short_term_ref_sets[0]; the harness asserts that case explicitly.
3. Same harness on the patched tree: all five cases (four adversarial
plus a legitimate-limit regression with ST=64, LT=32, num_neg=1)
pass clean, no KASAN reports.
If hardware-side validation actually does reject these counts before
rkvdec_hevc_assemble_hw_rps() runs and this patch is unnecessary, please
say so and I will withdraw it. If it is reachable, I will follow up with
a runtime hardware splat once the Orange Pi board I bought arrives.
Let me know if you want a patch set with the KUnit harnesses too.
checkpatch.pl: 0 errors / 0 warnings.
.../rockchip/rkvdec/rkvdec-hevc-common.c | 49 +++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
index 3119f3bc9f98..895fb16bc572 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
@@ -408,9 +408,58 @@ static void rkvdec_hevc_prepare_hw_st_rps(struct rkvdec_hevc_run *run, struct rk
memcpy(cache, run->ext_sps_st_rps, sizeof(struct v4l2_ctrl_hevc_ext_sps_st_rps));
}
+/*
+ * V4L2 caps the EXT_SPS RPS payload length but not the SPS-derived counts
+ * that the helpers walk. Caps match the HEVC spec ranges.
+ */
+#define RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS 64
+#define RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS 32
+#define RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS 16
+
+static int rkvdec_hevc_validate_rps_ctrls(struct rkvdec_hevc_run *run)
+{
+ const struct v4l2_ctrl_hevc_sps *sps = run->sps;
+
+ if (run->ext_sps_lt_rps &&
+ sps->num_long_term_ref_pics_sps >
+ RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS)
+ return -EINVAL;
+
+ if (run->ext_sps_st_rps) {
+ unsigned int i;
+
+ if (sps->num_short_term_ref_pic_sets >
+ RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS)
+ return -EINVAL;
+
+ for (i = 0; i < sps->num_short_term_ref_pic_sets; i++) {
+ const struct v4l2_ctrl_hevc_ext_sps_st_rps *r =
+ &run->ext_sps_st_rps[i];
+
+ if (r->num_negative_pics >
+ RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS ||
+ r->num_positive_pics >
+ RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS)
+ return -EINVAL;
+
+ if ((r->flags &
+ V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED) &&
+ (unsigned int)r->delta_idx_minus1 + 1 > i)
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
void rkvdec_hevc_assemble_hw_rps(struct rkvdec_hevc_run *run, struct rkvdec_rps *rps,
struct v4l2_ctrl_hevc_ext_sps_st_rps *st_cache)
{
+ if (rkvdec_hevc_validate_rps_ctrls(run)) {
+ pr_err_ratelimited("rkvdec: rejecting HEVC SPS/RPS controls with out-of-range counts\n");
+ return;
+ }
+
rkvdec_hevc_prepare_hw_st_rps(run, rps, st_cache);
rkvdec_hevc_assemble_hw_lt_rps(run, rps);
}