Re: [RFC PATCH 2/3] media: rockchip: add VEPU510 H.264 encoder driver for RK3576

[Heiko Stübner]

Am Mittwoch, 22. Juli 2026, 09:34:16 Mitteleuropäische Sommerzeit schrieb Jiaxing Hu:
> Add a from-scratch stateful V4L2 mem2mem driver for the Rockchip RK3576
> VEPU510 H.264 hardware video encoder (raw NV12 in, H.264 Annex-B out),
> modelled on the verisilicon/hantro device_run()/codec_ops split rather
> than the downstream MPP-service/task-queue model.
> 
> The two encoder cores (rkvenc0/rkvenc1) are exposed as two independent
> V4L2 M2M device nodes, each driving one physical core standalone; the
> downstream vendor CCU cross-core load-balancing is not implemented.
> 
> Register field semantics were worked out by trial-and-error against
> Rockchip's own open-source userspace codec library (rockchip-linux/mpp)
> and cross-checked against a real register-write trace of the downstream
> vendor stack running the same encode.
> 
> Intra (I-frame) encoding is confirmed working on real hardware (Radxa
> ROCK 4D): it produces valid H.264 that the reference decoders accept.
> Inter (P-frame) encoding still hits a hardware-watchdog stall in the
> reference-read datapath -- this is the main open question for this RFC;
> see the cover letter for the full symptom analysis.
> 
> Signed-off-by: Jiaxing Hu <gahing@xxxxxxxxxxxxx>

[...]

> diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
> new file mode 100644
> index 000000000..d70bcc5fb
> --- /dev/null
> +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
> @@ -0,0 +1,892 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Rockchip VEPU510 (RK3576) hardware video encoder driver.
> + *
> + * Copyright (C) 2026 Jiaxing Hu <gahing@xxxxxxxxxxxxx>
> + *
> + * Architecture notes (see also rkvenc-regs.h):
> + *
> + *  - This is a *stateful* V4L2 mem2mem encoder (raw NV12 in on OUTPUT,
> + *    H.264 Annex-B out on CAPTURE) modelled on
> + *    drivers/media/platform/verisilicon's hantro_drv.c device_run()/
> + *    codec_ops{run,done} split, NOT on rkvdec's stateless request-API
> + *    decoder pattern — an encoder has no per-frame bitstream to parse,
> + *    so there is nothing analogous to rkvdec_run_preamble/postamble here.
> + *
> + *  - The vendor downstream driver (rockchip-linux/kernel,
> + *    drivers/video/rockchip/mpp/mpp_rkvenc2.c) groups rkvenc0/rkvenc1
> + *    under a "CCU" (rockchip,rkv-encoder-rk3576-ccu) that does pure
> + *    software task-queue load balancing across both cores, plus an
> + *    optional DCHS (dual-core-handshake) register protocol used only
> + *    when *deliberately* splitting one frame's rows across both cores.
> + *    There is no hardware descriptor/link-list engine behind it (unlike
> + *    the decoder's CCU). v1 of this driver does not implement either:
> + *    rkvenc0 and rkvenc1 are exposed as two independent V4L2 M2M device
> + *    nodes, each driving one physical core standalone — the same
> + *    simplification rkvdec itself makes for multi-core VDPU hardware
> + *    (see rkvdec_disable_multicore()).

At least the comment and from a casual look also the code get this
backwards. The idea is to explicitly _not_ expose multiple video devices.

See
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/rockchip/rkvdec/rkvdec.c#n1613

Any (future) scheduling should happen inside the driver, and not get
offloaded onto _every_ userspace application individually.


Heiko