Re: [PATCH v6 2/2] arm64: dts: rockchip: Add Hantro G1 VPU support for RK3588

[Hugh Cole-Baker]

Hi Jianfeng,

> On 18 Apr 2024, at 12:10, Jianfeng Liu <liujianfeng1994@xxxxxxxxx> wrote:
> 
> Enable Hantro G1 video decoder in RK3588's devicetree.
> 
> Tested with FFmpeg v4l2_request code taken from [1]
> with MPEG2, H.264 and VP8 samples.
> 
> [1] https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch
> 
> Signed-off-by: Jianfeng Liu <liujianfeng1994@xxxxxxxxx>
> Tested-by: Hugh Cole-Baker <sigmaris@xxxxxxxxx>
> Reviewed-by: Sebastian Reichel <sebastian.reichel@xxxxxxxxxxxxx>
> ---
> arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 21 +++++++++++++++++++++
> 1 file changed, 21 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> index b0a59ec51..b0817382f 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
> @@ -1135,6 +1135,27 @@ power-domain@RK3588_PD_SDMMC {
> };
> };
> 
> + vpu: video-codec@fdb50000 {
> + compatible = "rockchip,rk3588-vdpu121", "rockchip,rk3568-vpu";
> + reg = <0x0 0xfdb50000 0x0 0x800>;

The register range at 0xfdb50000 length 0x800 includes "VEPU121 core0" encoder
regs at offset 0 and "VDPU121" decoder regs at offset 0x400 (referring to the
TRM v1.0 Part 1, section 5.5.1). So I think the "rockchip,rk3588-vdpu121"
compatible isn't exactly correct to use for this entire device.

IMO "rockchip,rk3588-vpu121" would be more appropriate if including both the
decoder and encoder. It also raises the question of whether the decoder and
encoder should be modeled in DT as one device like on RK3399, or separate
devices. In the vendor DT [0] they are modeled as two devices but they share
clocks, resets, IOMMU, and a "rockchip,taskqueue-node" value.

I've tested the JPEG encoding functionality of this encoder with [1], and it
seems to work, gstreamer produces a MJPEG video of the test pattern as
expected.

> + interrupts = <GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH 0>;
> + interrupt-names = "vdpu";
> + clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
> + clock-names = "aclk", "hclk";
> + iommus = <&vdpu_mmu>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + };
> +
> + vdpu_mmu: iommu@fdb50800 {
> + compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
> + reg = <0x0 0xfdb50800 0x0 0x40>;
> + interrupts = <GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH 0>;
> + clock-names = "aclk", "iface";
> + clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
> + power-domains = <&power RK3588_PD_VDPU>;
> + #iommu-cells = <0>;
> + };
> +
> av1d: video-codec@fdc70000 {
> compatible = "rockchip,rk3588-av1-vpu";
> reg = <0x0 0xfdc70000 0x0 0x800>;
> --
> 2.34.1
> 

[0]: https://github.com/friendlyarm/kernel-rockchip/blob/18fd1215fee01daef16b6ced1c0c3c3b83a4d8df/arch/arm64/boot/dts/rockchip/rk3588s.dtsi#L3630-L3683
[1]: https://github.com/sigmaris/linux/tree/rk3588-hantro-vpus
     with: gst-launch-1.0 videotestsrc pattern=ball flip=true ! v4l2jpegenc \
      ! matroskamux ! filesink location=jpegtest.mkv