Re: [PATCH 0/5] media: meson: vdec: Add VP9 decoding support

From: Neil Armstrong
Date: Fri Dec 13 2019 - 04:12:41 EST

Next message: Arnd Bergmann: "Re: [PATCH V2] MIPS: Add get_thread_area syscall"
Previous message: Stanley Chu: "[PATCH v1 4/4] scsi: ufs-mediatek: configure and enable clk-gating"
Next in thread: Nicolas Dufresne: "Re: [PATCH 0/5] media: meson: vdec: Add VP9 decoding support"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On 05/12/2019 19:46, Nicolas Dufresne wrote:
> Le jeudi 05 dÃcembre 2019 Ã 16:49 +0100, Neil Armstrong a Ãcrit :
>> On 05/12/2019 16:42, Nicolas Dufresne wrote:
>>> Hi Neil,
>>>
>>> Le jeudi 05 dÃcembre 2019 Ã 10:24 +0100, Neil Armstrong a Ãcrit :
>>>> Hello,
>>>>
>>>> This patchset aims to bring VP9 decoding support to Amlogic GXL, G12A & SM1
>>>> platforms for the amlogic stateful video decoder driver.
>>>>
>>>> With this, it passes v4l2-compliance with streaming on Amlogic G12A and
>>>> Amlogic SM1 SoCs successfully using the stream at [1] with a fixed
>>>> pyv4l2compliance script for VP9 at [2].
>>>>
>>>> The original script kept the IVF headers in the stream, confusing the
>>>> decoder. The fixed script only extracts the payload from the IVF container.
>>>>
>>>> The decoder has been tested using the Google CTS TestVectorsIttiam VP9 yuv420
>>>> samples, passing 82 resolutions test streams, with 13 fails by pixel
>>>> differences and 3 timeouts.
>>>
>>> How do you handle resolution changes on delta frames ? It's a bit of a
>>> challenge since the reference frames are not at the same resolution as
>>> the frames to be decoded. This breaks the assumption for the resolution
>>> changes mechanism as described in the spec.
>>
>> I don't have a lot of experience on the subject, but in the vendor implementation,
>> they store the resolution along the reference frames and when loading all the
>> reference frames to the HW, the original resolution is also loaded.
>> But we don't handle it.
>>
>>> On stateless side, Boris is introducing DESTROY_BUFS, so we can free
>>> the references when they are not used anymore. But the reference are
>>> managed by userspace and are not queued. While on stateful side so far,
>>> it was assumed that references are queued, and the semantic of S_FMT is
>>> that it must apply to the entire set of queued buffer.
>>
>> yes
>>
>>> I think most streams will work and won't use this feature, but I'm
>>> worried that writing a compliant VP9 decoder is currently not possible.
>>
>> Indeed, but I don't have a clear enough view on the subject, and it doesn't seem
>> I have any test stream with such feature.
>
> https://www.webmproject.org/vp9/levels/
>
> I believe in the performance suite, the FRM_RESIZE sample is what
> should exercise this.

Thanks for pointing this test suite... all samples were failing :-p

Now we added the probability handling and all samples works (except the unsupported level).
The ref handling was tricky aswell, anyway it's all fixed except FRM_RESIZE which will
be a hell of hacking to tackle.

The hack would be to keep the refs buffers we still need after frame resizing on drain,
then use the max(old_frame_size, new_frame_size) as capture buffer size, then copy back
the old refs buffers into the newly queued buffers and continue.

It's a ugly hack, but won't break the drain + S_FMT semantic.
But I will fix this _after_ the v2 of this serie is merged.

Neil

>
>>
>> Neil
>>
>>>> This patchset depends on :
>>>> - G12A enablement at [3]
>>>> - SM1 enablement at [4]
>>>> - H.264 and compliance at [5]
>>>>
>>>> [1] https://github.com/superna9999/pyv4l2compliance/raw/tests/output/Jellyfish_1080_10s_5MB.vp9.hdr
>>>> [2] https://github.com/superna9999/pyv4l2compliance
>>>> [3] https://lore.kernel.org/linux-media/20191120111430.29552-1-narmstrong@xxxxxxxxxxxx
>>>> [4] https://lore.kernel.org/linux-media/20191121101429.23831-1-narmstrong@xxxxxxxxxxxx
>>>> [5] https://lore.kernel.org/linux-media/20191126093733.32404-1-narmstrong@xxxxxxxxxxxx
>>>>
>>>> The compliance log is:
>>>> # v4l2-compliance --stream-from-hdr Jellyfish_1080_10s_5MB.vp9.hdr -s 200
>>>> v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
>>>>
>>>> Compliance test for meson-vdec device /dev/video0:
>>>>
>>>> Driver Info:
>>>> Driver name : meson-vdec
>>>> Card type : Amlogic Video Decoder
>>>> Bus info : platform:meson-vdec
>>>> Driver version : 5.4.0
>>>> Capabilities : 0x84204000
>>>> Video Memory-to-Memory Multiplanar
>>>> Streaming
>>>> Extended Pix Format
>>>> Device Capabilities
>>>> Device Caps : 0x04204000
>>>> Video Memory-to-Memory Multiplanar
>>>> Streaming
>>>> Extended Pix Format
>>>> Detected Stateful Decoder
>>>>
>>>> Required ioctls:
>>>> test VIDIOC_QUERYCAP: OK
>>>>
>>>> Allow for multiple opens:
>>>> test second /dev/video0 open: OK
>>>> test VIDIOC_QUERYCAP: OK
>>>> test VIDIOC_G/S_PRIORITY: OK
>>>> test for unlimited opens: OK
>>>>
>>>> Debug ioctls:
>>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>>
>>>> Input ioctls:
>>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>>
>>>> Output ioctls:
>>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>>
>>>> Input/Output configuration ioctls:
>>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>>
>>>> Control ioctls:
>>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>>> test VIDIOC_QUERYCTRL: OK
>>>> test VIDIOC_G/S_CTRL: OK
>>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>>> Standard Controls: 2 Private Controls: 0
>>>>
>>>> Format ioctls:
>>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>>> test VIDIOC_G_FMT: OK
>>>> test VIDIOC_TRY_FMT: OK
>>>> test VIDIOC_S_FMT: OK
>>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>>> test Cropping: OK (Not Supported)
>>>> test Composing: OK (Not Supported)
>>>> test Scaling: OK (Not Supported)
>>>>
>>>> Codec ioctls:
>>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>> test VIDIOC_(TRY_)DECODER_CMD: OK
>>>>
>>>> Buffer ioctls:
>>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>>>> test VIDIOC_EXPBUF: OK
>>>> test Requests: OK (Not Supported)
>>>>
>>>> Test input 0:
>>>>
>>>> Streaming ioctls:
>>>> test read/write: OK (Not Supported)
>>>> test blocking wait: OK
>>>> Video Capture Multiplanar: Captured 200 buffers
>>>> test MMAP (select): OK
>>>> Video Capture Multiplanar: Captured 200 buffers
>>>> test MMAP (epoll): OK
>>>> test USERPTR (select): OK (Not Supported)
>>>> test DMABUF: Cannot test, specify --expbuf-device
>>>>
>>>> Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
>>>>
>>>> Maxime Jourdan (4):
>>>> media: meson: vdec: add helpers for lossless framebuffer compression
>>>> buffers
>>>> media: meson: vdec: add common HEVC decoder support
>>>> media: meson: vdec: add VP9 input support
>>>> media: meson: vdec: add VP9 decoder support
>>>>
>>>> Neil Armstrong (1):
>>>> media: meson: vdec: align stride on 32 bytes
>>>>
>>>> drivers/staging/media/meson/vdec/Makefile | 4 +-
>>>> .../media/meson/vdec/codec_hevc_common.c | 285 ++++
>>>> .../media/meson/vdec/codec_hevc_common.h | 77 ++
>>>> drivers/staging/media/meson/vdec/codec_vp9.c | 1192 +++++++++++++++++
>>>> drivers/staging/media/meson/vdec/codec_vp9.h | 13 +
>>>> drivers/staging/media/meson/vdec/esparser.c | 142 +-
>>>> drivers/staging/media/meson/vdec/hevc_regs.h | 218 +++
>>>> drivers/staging/media/meson/vdec/vdec.c | 10 +-
>>>> .../staging/media/meson/vdec/vdec_helpers.c | 31 +-
>>>> .../staging/media/meson/vdec/vdec_helpers.h | 4 +
>>>> drivers/staging/media/meson/vdec/vdec_hevc.c | 231 ++++
>>>> drivers/staging/media/meson/vdec/vdec_hevc.h | 13 +
>>>> .../staging/media/meson/vdec/vdec_platform.c | 38 +
>>>> 13 files changed, 2245 insertions(+), 13 deletions(-)
>>>> create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.c
>>>> create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.h
>>>> create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.c
>>>> create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.h
>>>> create mode 100644 drivers/staging/media/meson/vdec/hevc_regs.h
>>>> create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.c
>>>> create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.h
>>>>
>>
>>

Attachment: signature.asc
Description: OpenPGP digital signature

Next message: Arnd Bergmann: "Re: [PATCH V2] MIPS: Add get_thread_area syscall"
Previous message: Stanley Chu: "[PATCH v1 4/4] scsi: ufs-mediatek: configure and enable clk-gating"
Next in thread: Nicolas Dufresne: "Re: [PATCH 0/5] media: meson: vdec: Add VP9 decoding support"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]