Re: [PATCH v1 0/9] AV1 stateless decoder for RK3588

From: Michael Grzeschik
Date: Mon Dec 19 2022 - 16:54:56 EST


Hi Benjamin,
Hi Ezequiel,

On Mon, Dec 19, 2022 at 06:07:38PM -0300, Ezequiel Garcia wrote:
On Mon, Dec 19, 2022 at 12:56 PM Benjamin Gaignard
<benjamin.gaignard@xxxxxxxxxxxxx> wrote:

This series implement AV1 stateless decoder for RK3588 SoC.
The harware support 8 and 10 bits bitstreams up to 7680x4320.
AV1 feature like film grain or scaling are done by the postprocessor.
The driver can produce NV12_4L4 and NV12 pixel formats.
A native 10bits NV12_4L4 format is possible but need more investigation
to be completly documented and enabled.

It is based on Daniel's "[RFC,v3] media: Add AV1 uAPI" [1] patches and
Sebastian's device-tree patches for RK3588.


I thought the AV1 decoder in RK3588 was really a separate hardware
from the Hantro G1/G2.

Shouldn't this need a new driver for this new hardware?

Just jumping into this discussion as I am currently working on the rkvenc driver.

In my case I am extending the rkvdec driver to become more generic for
other rockchip specific enc/decoders.

My first change looks like this:
---
drivers/staging/media/rkvdec/Makefile | 4 +-
drivers/staging/media/rkvdec/rkvdec-h264.c | 100 ++++-----
drivers/staging/media/rkvdec/rkvdec-vp9.c | 142 ++++++-------
drivers/staging/media/rkvdec/{rkvdec.c => rkvpu.c} | 510 +++++++++++++++++++++++-----------------------
drivers/staging/media/rkvdec/{rkvdec.h => rkvpu.h} | 66 +++---
---

While working on other parts of the encoder I found many places in the
rkvdec driver (e.g. v4l2 and vb2 callbacks) that looked familiar to the hantro
functions but where limited to the decoder case.

I think there are two options for the av1 codec.

1) If the vpu981 is a driver that has nothing to do with verisilicon but
works with this driver framework, then we should integrate vepu981 into it
but consider rename the verisilicon unrelated parts to something generic.

2) Move the vepu981 av1 driver into the rkvdec instead.

If 1) is the way to go, we can even think of moving the staging code parts from
rkvdec to the verisilicon code. Likewise to the vepu981-av1.

I could also keep on integrating the rkvenc on that base instead.

Regards,
Michael

The full branch can be found here:
https://gitlab.collabora.com/linux/for-upstream/-/commits/rk3588_av1_decoder_v1

Fluster score is: 151/239 while testing AV1-TEST-VECTORS with GStreamer-AV1-V4L2SL-Gst1.0.
The failing tests are:
- 10bits bitstream because 10bits output formats aren't yet implemented.
- the 2 tests with 2 spatial layers: few errors in luma/chroma values
- tests with resolution < hardware limit (64x64)

Benjamin

Benjamin Gaignard (9):
dt-bindings: media: rockchip-vpu: Add rk3588 vpu compatible
media: verisilicon: Add AV1 decoder mode and controls
media: verisilicon: Save bit depth for AV1 decoder
media: verisilicon: Check AV1 bitstreams bit depth
media: verisilicon: Compute motion vectors size for AV1 frames
media: verisilicon: Add AV1 entropy helpers
media: verisilicon: Add Rockchip AV1 decoder
media: verisilicon: Add film grain feature to AV1 driver
media: verisilicon: Enable AV1 decoder on rk3588

.../bindings/media/rockchip-vpu.yaml | 1 +
drivers/media/platform/verisilicon/Makefile | 3 +
drivers/media/platform/verisilicon/hantro.h | 5 +
.../media/platform/verisilicon/hantro_drv.c | 54 +
.../media/platform/verisilicon/hantro_hw.h | 102 +
.../platform/verisilicon/hantro_postproc.c | 3 +
.../media/platform/verisilicon/hantro_v4l2.c | 5 +
.../verisilicon/rockchip_av1_entropymode.c | 4536 +++++++++++++++++
.../verisilicon/rockchip_av1_entropymode.h | 272 +
.../verisilicon/rockchip_av1_filmgrain.c | 401 ++
.../verisilicon/rockchip_av1_filmgrain.h | 36 +
.../verisilicon/rockchip_vpu981_hw_av1_dec.c | 2280 +++++++++
.../verisilicon/rockchip_vpu981_regs.h | 477 ++
.../platform/verisilicon/rockchip_vpu_hw.c | 116 +
14 files changed, 8291 insertions(+)
create mode 100644 drivers/media/platform/verisilicon/rockchip_av1_entropymode.c
create mode 100644 drivers/media/platform/verisilicon/rockchip_av1_entropymode.h
create mode 100644 drivers/media/platform/verisilicon/rockchip_av1_filmgrain.c
create mode 100644 drivers/media/platform/verisilicon/rockchip_av1_filmgrain.h
create mode 100644 drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
create mode 100644 drivers/media/platform/verisilicon/rockchip_vpu981_regs.h

--
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |

Attachment: signature.asc
Description: PGP signature