[PATCH 2/2] media: qcom: iris: venc: reject encoder crop rectangles larger than the source
From: Loic Poulain
Date: Thu Nov 12 2026 - 11:27:04 EST
iris_venc_s_selection() bounded the V4L2_SEL_TGT_CROP rectangle by
inst->fmt_src, the padded buffer geometry, instead of the visible
resolution in enc_raw_width and enc_raw_height. fmt_src is rounded up, so
userspace could set a crop larger than the frame it submits.
iris_hfi_gen2_set_crop_offsets() then underflows
bottom_offset = inst->enc_raw_height - inst->crop.height;
and the encode fails silently. All ioctls succeed and the EOS event is
delivered, but the encoder produces no data and nothing is logged.
Bound the rectangle by enc_raw_* instead.
v4l2-ctl -d /dev/video1 \
--set-fmt-video-out=width=1280,height=720,pixelformat=NV12 \
--set-selection-output=target=crop,width=1280,height=730 \
--stream-mmap --stream-out-mmap --stream-count=30 \
--stream-to=out.h264
Before, 1280x730 is accepted because fmt_src is 1280x736, and out.h264 is
0 bytes. After:
VIDIOC_S_SELECTION: failed: Invalid argument
Rectangles within the source, like 1280x704 and 640x360, are still
accepted.
Fixes: d9967fa37ca9 ("media: qcom: iris: Improve format alignment for encoder")
Signed-off-by: Loic Poulain <loic.poulain@xxxxxxxxxxxxxxxx>
---
drivers/media/platform/qcom/iris/iris_venc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/qcom/iris/iris_venc.c b/drivers/media/platform/qcom/iris/iris_venc.c
index b367367ac2ccf874a00606b8b1352fed37b37a57..1af692d10bcf77060fa95c6c3906e9067d167d84 100644
--- a/drivers/media/platform/qcom/iris/iris_venc.c
+++ b/drivers/media/platform/qcom/iris/iris_venc.c
@@ -380,8 +380,8 @@ int iris_venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s)
s->r.left = 0;
s->r.top = 0;
- if (s->r.width > inst->fmt_src->fmt.pix_mp.width ||
- s->r.height > inst->fmt_src->fmt.pix_mp.height)
+ if (s->r.width > inst->enc_raw_width ||
+ s->r.height > inst->enc_raw_height)
return -EINVAL;
inst->crop.left = s->r.left;
--
2.34.1