[PATCH 0/8] media: mediatek: Fix MT8173 decoding through GStreamer

From: Ryan Brue

Date: Fri Nov 20 2026 - 14:55:47 EST


These fix the MT8173 video decoder and MDP for clients that follow the
stateful decoder interface, found while getting GStreamer's v4l2 decoders
to play through them.

Patch 1 is the main one: the MT8173 VPU firmware takes one H.264 NAL unit
per call, but the interface hands the driver whole access units, so any
access unit that begins with a delimiter, parameter sets or an SEI failed,
and pictures of more than one slice never decoded at all. The driver now
walks the units of each buffer. Buffers holding one NAL unit decode as
before.

Patches 2 and 3 fix the decoder's answers to what GStreamer asks: a zero
bitstream buffer size, and the default compose rectangle (1088 lines for a
1080-line H.264 stream).

Patches 4 to 7 fix the MDP: MT21C sources that are not whole 16x32 tiles,
which the firmware refuses (1280x720 failed on every frame), a crop
refused against the destination size a client had not set yet (the
scaling ratio is now checked once both queues stream), buffer plane
sizes that were never checked, and a format change that took effect even
when it was refused. Patch 8 fixes the same last bug in the decoder,
where VIDIOC_S_FMT on a busy queue changed the format and returned 0.

Patch 1 carries no Fixes tag: the driver has taken one NAL unit per buffer
since it was added, and the change is larger than a stable fix usually is.

Tested on an Amazon Fire HD 10 (2017), an MT8173 tablet (board DTS under
review), on v7.4-rc2 with VPU firmware 1.2.0 and Youngjoon Lee's MDP RFC
[1], without which the MDP writes nothing on this board. Through a small
V4L2 test client with one access unit per buffer, the decoder and the
MDP turn H.264 Baseline, Main and High, H.264 with four slices per
picture, VP8 and VP9 into frames that match ffmpeg's software decode
byte for byte. With one NAL unit per buffer the output is byte-exact as
before, for every stream but the four-slice one, which cannot decode
that way. Through GStreamer's v4l2 decoders, H.264 High and VP9 also
match ffmpeg byte for byte, every frame; 1080p H.264 and VP9 play
through GStreamer as well.

v4l2-compliance 1.32.0: the MDP passes 48 of 48 with the series (47
without patch 6). A downscale beyond the MDP's limits (1280x720 to
16x16) is refused at VIDIOC_STREAMON; a 2:1 one streams. The decoder
passes 47 of 48; the failure is in the controls test, about
V4L2_CID_MIN_BUFFERS_FOR_CAPTURE, in code the series does not touch.
Only MT8173 was tested; patches 2, 3 and 8 touch code the later SoCs
share. Patch 8 makes the stateless decoders refuse VIDIOC_S_FMT on a
busy CAPTURE queue, as the spec asks; confirmation that ChromeOS's
clients free those buffers first would be welcome.

Not fixed here: a resolution change in the middle of a stream (dynamic
resolution change, dev-decoder.rst) stalls on MT8173. The decoder signals
V4L2_EVENT_SOURCE_CHANGE but never returns the CAPTURE buffer marked
V4L2_BUF_FLAG_LAST, so a client waiting for it to reallocate waits for
ever. Stock v7.4-rc2 behaves the same with one NAL unit per buffer, and
the series doesn't change it. That's for a separate fix.

This series was developed with the help of an LLM.

[1] https://lore.kernel.org/all/20261111-rfc-v2-generated-v3-1-e49697d20806@xxxxxxxxx/

Signed-off-by: Ryan Brue <ryanbrue.dev@xxxxxxxxx>
---
Ryan Brue (8):
media: mediatek: vcodec: Decode H.264 access units on MT8173
media: mediatek: vcodec: Pick an OUTPUT buffer size when given zero
media: mediatek: vcodec: Report the visible rectangle as default compose
media: mtk-mdp: Round MT21C sources up to whole tiles
media: mtk-mdp: Do not refuse a crop for the other side's size
media: mtk-mdp: Check buffer plane sizes against the format
media: mtk-mdp: Refuse a format change once buffers are allocated
media: mediatek: vcodec: Refuse a format change once buffers are allocated

drivers/media/platform/mediatek/mdp/mtk_mdp_m2m.c | 148 +++++++++++-------
.../mediatek/vcodec/decoder/mtk_vcodec_dec.c | 58 ++++---
.../mediatek/vcodec/decoder/vdec/vdec_h264_if.c | 168 ++++++++++++++++++++-
3 files changed, 287 insertions(+), 87 deletions(-)
---
base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb
change-id: 20261113-rbrue-suez-upstreaming-mtk-vcodec-mdp-fixes-8e7b09b7663f

Best regards,
--
Ryan Brue <ryanbrue.dev@xxxxxxxxx>