[PATCH v2 0/3] media: qcom: iris: venc: fix encoder crop handling

From: Loic Poulain

Date: Tue Nov 17 2026 - 05:47:11 EST


The encoder keeps the source resolution in two places. inst->fmt_src holds
the padded buffer geometry reported to userspace, with width aligned to 128
and height to 32. enc_raw_width and enc_raw_height hold the visible
resolution sent to the firmware.

inst->crop and inst->enc_scale_* belong to the second space, because
iris_hfi_gen2_set_crop_offsets() computes the offsets as enc_raw_* minus
crop. Each patch fixes a place where the padded size was used instead.

Patch 1 seeds inst->crop from the visible size. Before, a plain 1280x720
session ended up with crop.height 736 and a bottom offset of 0xfffffff0.

Patch 2 compares the requested coded size against the visible resolution,
so a plain 1280x720 session is no longer taken for a 736 -> 720 downscale.
fmt_dst now also carries the codec alignment on every path, so the CAPTURE
queue keeps reporting the coded size.

Patch 3 bounds the S_SELECTION rectangle by the visible size, so userspace
can no longer select a crop larger than the frame it submits and silently
get an empty bitstream.

Tested on QCS8300 with v4l2-ctl and ffmpeg.

Signed-off-by: Loic Poulain <loic.poulain@xxxxxxxxxxxxxxxx>
---
Changes in v2:
- Split the crop fix and the scaling fix into two separate patches, as
suggested by Bryan.
- Trim the commit message of the crop patch, the HFI offset computation is
enough to explain the bug.
- Link to v1: https://lore.kernel.org/r/20261112-iris-fixes-v1-0-22b759c3c285@xxxxxxxxxxxxxxxx

---
Loic Poulain (3):
media: qcom: iris: venc: keep the crop rectangle in visible resolution space
media: qcom: iris: venc: detect scaling against the visible resolution
media: qcom: iris: venc: reject encoder crop rectangles larger than the source

drivers/media/platform/qcom/iris/iris_venc.c | 24 ++++++++++++++----------
drivers/media/platform/qcom/iris/iris_vidc.c | 4 ++--
2 files changed, 16 insertions(+), 12 deletions(-)
---
base-commit: e726cf08ee6ba7e8b852c77760f12c59f90f1338
change-id: 20261112-iris-fixes-f6130f20caf9

Best regards,
--
Loic Poulain <loic.poulain@xxxxxxxxxxxxxxxx>