[PATCH v2 0/2] media: rockchip: Decode monochrome H.264 to GREY
From: Arun Iyengar via B4 Relay
Date: Sat Nov 21 2026 - 09:25:38 EST
The Rockchip Hantro (G1/VDPU2) and rkvdec decoders write no chroma for
4:0:0 H.264, but the drivers offer only formats with chroma. Offer GREY
for such streams.
Tested on an Orange Pi 5 (RK3588S: VDPU121 and rkvdec) on a 7.3-based
tree that carries my other pending Hantro series; build-tested on media
next. v1 passed fluster: JVT-FR-EXT HPCAMOLQ_BRCM_B and HPCVMOLQ_BRCM_B
on both decoders, other results unchanged. v2: format and control ioctl
cases, including a too small CAPTURE buffer and a refused request, with
the size check and request error path of patch 1 in the posted form;
4:0:0, 4:2:0 and concatenated streams bit-exact with a locally patched
GStreamer, there with the size check of [2]. All under KASAN. Other
SoCs are untested. Userspace without GREY support gets an error for
these streams instead of wrong chroma.
A change between 4:0:0 and 4:2:0 needs new CAPTURE buffers: Hantro
refuses the new SPS with -EBUSY while CAPTURE streams, rkvdec while
CAPTURE buffers exist (as before; the rkvdec concatenation ran only
with an out-of-tree busy check).
Conflicts with my pending [1] in hantro_try_fmt() and [2] in
device_run() (the same size check); I will rebase whichever goes second.
[1] https://lore.kernel.org/r/20261112-hantro-fmt-role-v1-1-e596d6f4828f@xxxxxxxxx
[2] https://lore.kernel.org/r/20261112-hantro-dec-size-check-v1-1-e5f1180b625f@xxxxxxxxx
Developed with Claude (Anthropic), which wrote the code and the commit
messages. I reviewed every change.
---
Changes in v2:
- Hantro: GREY only for High profile 4:0:0, as the hardware does (Sashiko)
- Hantro: no GREY on open, SPS ignored for other codecs (Sashiko)
- Hantro: keep the 4:0:0 state across S_FMT(OUTPUT) (Sashiko)
- Hantro: no transient 4:0:0 state on variants without GREY (Sashiko)
- Hantro: fail refused requests and too small CAPTURE buffers (Sashiko)
- Link to v1: https://patch.msgid.link/20261120-b4-h264-monochrome-grey-v1-0-887102a688fd@xxxxxxxxx
---
Arun Iyengar (2):
media: verisilicon: Decode monochrome H.264 to GREY on Rockchip
media: rkvdec: Decode monochrome H.264 to GREY
.../platform/rockchip/rkvdec/rkvdec-h264-common.c | 4 +-
drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 ++
drivers/media/platform/rockchip/rkvdec/rkvdec.h | 1 +
drivers/media/platform/verisilicon/hantro.h | 11 ++++
drivers/media/platform/verisilicon/hantro_drv.c | 71 ++++++++++++++++++++--
drivers/media/platform/verisilicon/hantro_h264.c | 11 +++-
drivers/media/platform/verisilicon/hantro_hw.h | 2 +-
drivers/media/platform/verisilicon/hantro_v4l2.c | 22 +++++--
.../media/platform/verisilicon/rockchip_vpu_hw.c | 39 ++++++++++++
9 files changed, 152 insertions(+), 13 deletions(-)
---
base-commit: ccf78740d63813c452c035b57d17a650492562fa
change-id: 20261114-b4-h264-monochrome-grey-7d1fabf34027
Best regards,
--
Arun Iyengar <aiyengar1990@xxxxxxxxx>