Re: [PATCH 0/5] media: meson: vdec: Add VP9 decoding support

[Neil Armstrong]

On 05/12/2019 16:42, Nicolas Dufresne wrote:
> Hi Neil,
> 
> Le jeudi 05 dÃcembre 2019 Ã 10:24 +0100, Neil Armstrong a Ãcrit :
>> Hello,
>>
>> This patchset aims to bring VP9 decoding support to Amlogic GXL, G12A & SM1
>> platforms for the amlogic stateful video decoder driver.
>>
>> With this, it passes v4l2-compliance with streaming on Amlogic G12A and
>> Amlogic SM1 SoCs successfully using the stream at [1] with a fixed
>> pyv4l2compliance script for VP9 at [2].
>>
>> The original script kept the IVF headers in the stream, confusing the
>> decoder. The fixed script only extracts the payload from the IVF container.
>>
>> The decoder has been tested using the Google CTS TestVectorsIttiam VP9 yuv420
>> samples, passing 82 resolutions test streams, with 13 fails by pixel
>> differences and 3 timeouts.
> 
> How do you handle resolution changes on delta frames ? It's a bit of a
> challenge since the reference frames are not at the same resolution as
> the frames to be decoded. This breaks the assumption for the resolution
> changes mechanism as described in the spec.

I don't have a lot of experience on the subject, but in the vendor implementation,
they store the resolution along the reference frames and when loading all the
reference frames to the HW, the original resolution is also loaded.
But we don't handle it.

> 
> On stateless side, Boris is introducing DESTROY_BUFS, so we can free
> the references when they are not used anymore. But the reference are
> managed by userspace and are not queued. While on stateful side so far,
> it was assumed that references are queued, and the semantic of S_FMT is
> that it must apply to the entire set of queued buffer.

yes

> 
> I think most streams will work and won't use this feature, but I'm
> worried that writing a compliant VP9 decoder is currently not possible.

Indeed, but I don't have a clear enough view on the subject, and it doesn't seem
I have any test stream with such feature.

Neil

> 
>>
>> This patchset depends on :
>> - G12A enablement at [3]
>> - SM1 enablement at [4]
>> - H.264 and compliance at [5]
>>
>> [1] https://github.com/superna9999/pyv4l2compliance/raw/tests/output/Jellyfish_1080_10s_5MB.vp9.hdr
>> [2] https://github.com/superna9999/pyv4l2compliance
>> [3] https://lore.kernel.org/linux-media/20191120111430.29552-1-narmstrong@xxxxxxxxxxxx
>> [4] https://lore.kernel.org/linux-media/20191121101429.23831-1-narmstrong@xxxxxxxxxxxx
>> [5] https://lore.kernel.org/linux-media/20191126093733.32404-1-narmstrong@xxxxxxxxxxxx
>>
>> The compliance log is:
>> # v4l2-compliance --stream-from-hdr Jellyfish_1080_10s_5MB.vp9.hdr -s 200
>> v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
>>
>> Compliance test for meson-vdec device /dev/video0:
>>
>> Driver Info:
>> 	Driver name      : meson-vdec
>> 	Card type        : Amlogic Video Decoder
>> 	Bus info         : platform:meson-vdec
>> 	Driver version   : 5.4.0
>> 	Capabilities     : 0x84204000
>> 		Video Memory-to-Memory Multiplanar
>> 		Streaming
>> 		Extended Pix Format
>> 		Device Capabilities
>> 	Device Caps      : 0x04204000
>> 		Video Memory-to-Memory Multiplanar
>> 		Streaming
>> 		Extended Pix Format
>> 	Detected Stateful Decoder
>>
>> Required ioctls:
>> 	test VIDIOC_QUERYCAP: OK
>>
>> Allow for multiple opens:
>> 	test second /dev/video0 open: OK
>> 	test VIDIOC_QUERYCAP: OK
>> 	test VIDIOC_G/S_PRIORITY: OK
>> 	test for unlimited opens: OK
>>
>> Debug ioctls:
>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>
>> Input ioctls:
>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>
>> Output ioctls:
>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>
>> Input/Output configuration ioctls:
>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>
>> Control ioctls:
>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>> 	test VIDIOC_QUERYCTRL: OK
>> 	test VIDIOC_G/S_CTRL: OK
>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>> 	Standard Controls: 2 Private Controls: 0
>>
>> Format ioctls:
>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>> 	test VIDIOC_G_FMT: OK
>> 	test VIDIOC_TRY_FMT: OK
>> 	test VIDIOC_S_FMT: OK
>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>> 	test Cropping: OK (Not Supported)
>> 	test Composing: OK (Not Supported)
>> 	test Scaling: OK (Not Supported)
>>
>> Codec ioctls:
>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>> 	test VIDIOC_(TRY_)DECODER_CMD: OK
>>
>> Buffer ioctls:
>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>> 	test VIDIOC_EXPBUF: OK
>> 	test Requests: OK (Not Supported)
>>
>> Test input 0:
>>
>> Streaming ioctls:
>> 	test read/write: OK (Not Supported)
>> 	test blocking wait: OK
>> 	Video Capture Multiplanar: Captured 200 buffers   
>> 	test MMAP (select): OK
>> 	Video Capture Multiplanar: Captured 200 buffers   
>> 	test MMAP (epoll): OK
>> 	test USERPTR (select): OK (Not Supported)
>> 	test DMABUF: Cannot test, specify --expbuf-device
>>
>> Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
>>
>> Maxime Jourdan (4):
>>   media: meson: vdec: add helpers for lossless framebuffer compression
>>     buffers
>>   media: meson: vdec: add common HEVC decoder support
>>   media: meson: vdec: add VP9 input support
>>   media: meson: vdec: add VP9 decoder support
>>
>> Neil Armstrong (1):
>>   media: meson: vdec: align stride on 32 bytes
>>
>>  drivers/staging/media/meson/vdec/Makefile     |    4 +-
>>  .../media/meson/vdec/codec_hevc_common.c      |  285 ++++
>>  .../media/meson/vdec/codec_hevc_common.h      |   77 ++
>>  drivers/staging/media/meson/vdec/codec_vp9.c  | 1192 +++++++++++++++++
>>  drivers/staging/media/meson/vdec/codec_vp9.h  |   13 +
>>  drivers/staging/media/meson/vdec/esparser.c   |  142 +-
>>  drivers/staging/media/meson/vdec/hevc_regs.h  |  218 +++
>>  drivers/staging/media/meson/vdec/vdec.c       |   10 +-
>>  .../staging/media/meson/vdec/vdec_helpers.c   |   31 +-
>>  .../staging/media/meson/vdec/vdec_helpers.h   |    4 +
>>  drivers/staging/media/meson/vdec/vdec_hevc.c  |  231 ++++
>>  drivers/staging/media/meson/vdec/vdec_hevc.h  |   13 +
>>  .../staging/media/meson/vdec/vdec_platform.c  |   38 +
>>  13 files changed, 2245 insertions(+), 13 deletions(-)
>>  create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.c
>>  create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.h
>>  create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.c
>>  create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.h
>>  create mode 100644 drivers/staging/media/meson/vdec/hevc_regs.h
>>  create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.c
>>  create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.h
>>

Attachment: signature.asc
Description: OpenPGP digital signature