RE: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder driver

[Ming Qian]

> -----Original Message-----
> From: Nicolas Dufresne [mailto:nicolas@xxxxxxxxxxxx]
> Sent: Wednesday, November 24, 2021 10:58 PM
> To: Ming Qian <ming.qian@xxxxxxx>; mchehab@xxxxxxxxxx;
> shawnguo@xxxxxxxxxx; robh+dt@xxxxxxxxxx; s.hauer@xxxxxxxxxxxxxx
> Cc: hverkuil-cisco@xxxxxxxxx; kernel@xxxxxxxxxxxxxx; festevam@xxxxxxxxx;
> dl-linux-imx <linux-imx@xxxxxxx>; Aisheng Dong <aisheng.dong@xxxxxxx>;
> linux-media@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> devicetree@xxxxxxxxxxxxxxx; linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder
> driver
> 
> Caution: EXT Email
> 
> Le mercredi 24 novembre 2021 à 09:00 +0000, Ming Qian a écrit :
> > > -----Original Message-----
> > > From: Nicolas Dufresne [mailto:nicolas@xxxxxxxxxxxx]
> > > Sent: Wednesday, November 24, 2021 3:23 AM
> > > To: Ming Qian <ming.qian@xxxxxxx>; mchehab@xxxxxxxxxx;
> > > shawnguo@xxxxxxxxxx; robh+dt@xxxxxxxxxx; s.hauer@xxxxxxxxxxxxxx
> > > Cc: hverkuil-cisco@xxxxxxxxx; kernel@xxxxxxxxxxxxxx;
> > > festevam@xxxxxxxxx; dl-linux-imx <linux-imx@xxxxxxx>; Aisheng Dong
> > > <aisheng.dong@xxxxxxx>; linux-media@xxxxxxxxxxxxxxx;
> > > linux-kernel@xxxxxxxxxxxxxxx; devicetree@xxxxxxxxxxxxxxx;
> > > linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
> > > Subject: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder
> > > driver
> > >
> > > Caution: EXT Email
> > >
> > > Hi Ming,
> > >
> > > For the patchset:
> > >
> > > Tested-By: Nicolas Dufresne <nicolas.dufresne@xxxxxxxxxxxxx>
> > >
> > > I've validated encoding manually with GStreamer:
> > >
> > >   gst-launch-1.0 videotestsrc num-buffers=100 ! v4l2h264enc !
> > > video/x-h264,profile=main,level=\(string\)2 ! h264parse ! qtmux !
> > > filesink location=test.mov
> > >
> > > I've also verify the that the number of frames was exactly 100, this
> > > is common issue with V4L2 encoders. Then I have validated VP8, H.264
> > > and H.265 decoders using fluster [0] with this PR [1] applied. You
> > > can find full summary at the end of this email. Markdown report
> > > didn't get generated, I will have to check with upstream fluster if
> > > there is a regression.
> > >
> > > $> ./fluster.py run -s -so amphion-imx8qxp-conformance.md -d
> > > GStreamer-VP8-V4L2-Gst1.0 GStreamer-H.264-V4L2-Gst1.0
> > > GStreamer-H.265-V4L2-Gst1.0
> > >
> > > VP8:   Ran 59/61 tests successfully               in 131.788 secs
> > > H.264: Ran 75/135 tests successfully               in 501.206 secs
> > > H.265: Ran 126/147 tests successfully               in 1131.933 secs
> > >
> > > Note that in mainline, only 1 core get fired and is kept at its
> > > lowest possible frequency, so perhaps it may cause some of the
> > > timeout seen. The driver is overall functional, and I would like to
> > > thank you for this extra work. Also, note that this very first time
> > > I run Fluster over the stateful CODEC wrappers. I will need to run
> > > this on more platforms to locate the GStreamer specific fail.
> > >
> > > VP8 note, conformance vector vp80-03-segmentation-1425 cause a hang
> > > but it then
> > > recover:
> > >
> > > [ 8264.851841] amphion-vpu-core 2d040000.vpu_core: [0] sync session
> > > timeout [ 8264.858634] amphion-vpu-core 2d040000.vpu_core: [0] send
> > > cmd(0x2) fail [ 8264.867992] amphion-vpu-core 2d040000.vpu_core: [0]
> > > start fail [ 8264.905173] amphion-vpu-core 2d040000.vpu_core: reset
> > > hang core
> > >
> >
> > HI Nicolas
> >
> >     There is a bug in firmware that if send a command to firmware too
> > close after stop cmd, The firmware may enter wfi wrong, and led to
> > hang issue you met in vp80-03- segmentation-1425.
> > I'll add a workaround in driver that add a delay after send stop cmd
> > to firmware in next version.
> >
> >     Because the amphion's vpu doesn't support to output i420, so the
> > test will convert nv12_8l128 to i420 by videoconvert, it leds to most
> > of timeout failure.
> >
> >     The FM1_BT_B.h264 can't be decoded by amphion's vpu, the vpu is
> > keeping parse sequence header, and it led to timeout failure.
> >
> >     I run the test and change the timeout to 300, then most of timeout
> > failures are gone. Besides that, my result is almost as same as yours.
> 
> Oh my bad, I forgot about the short timeout, with a single core on top of all
> this, that makes sense.
> 
> >
> >     The failures of assertion error means that the vpu's output is
> > different from the pattern, I think it should be the vpu's limitation.
> 
> Most likely, best way to know is to keep the results (--keep) and visually look at
> the result. My expectation with this is that we get decent results and that none
> of the issue render the VPU or the system unusable. Each company is then
> responsible for their CODEC conformance, specially with stateful, there is very
> little that userspace will be responsible with. Though if you do find issue that is
> clearly caused by GStreaner, let me know, I'll be more then happy to fix.
> Most VPU providers will also buy proprietary conformance suite (like Allegro),
> which covers much more then basic conformance.
> 

For test [JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0) VPSSPSPPS_A_MainConcept_1,
The vpu report an unsupported message to driver, so driver report pollerr to gstreamer.
But this stream can be decoded using the amphion vpu when I test it using our unit test,
I checked the difference, there are many vps, sps and pps at the beginning, 
gstreamer will skip the first vpu and two pps, totally skip 56 bytes. It leds to vpu can't decode
And our unit test won't skip anthing, so the vpu can decode the stream.