Re: [PATCH] media: hantro: Support H264 profile control

[Tomasz Figa]

On Sat, Nov 23, 2019 at 1:52 AM Nicolas Dufresne <nicolas@xxxxxxxxxxxx> wrote:
>
> Le samedi 23 novembre 2019 Ã 01:00 +0900, Tomasz Figa a Ãcrit :
> > On Sat, Nov 23, 2019 at 12:09 AM Nicolas Dufresne <nicolas@xxxxxxxxxxxx> wrote:
> > > Le vendredi 22 novembre 2019 Ã 14:16 +0900, Hirokazu Honda a Ãcrit :
> > > > The Hantro G1 decoder supports H.264 profiles from Baseline to High, with
> > > > the exception of the Extended profile.
> > > >
> > > > Expose the V4L2_CID_MPEG_VIDEO_H264_PROFILE control, so that the
> > > > applications can query the driver for the list of supported profiles.
> > >
> > > Thanks for this patch. Do you think we could also add the LEVEL control
> > > so the profile/level enumeration becomes complete ?
> > >
> > > I'm thinking it would be nice if the v4l2 compliance test make sure
> > > that codecs do implement these controls (both stateful and stateless),
> > > it's essential for stack with software fallback, or multiple capable
> > > codec hardware but with different capabilities.
> > >
> >
> > Level is a difficult story, because it also specifies the number of
> > macroblocks per second, but for decoders like this the number of
> > macroblocks per second it can handle depends on things the driver
> > might be not aware of - clock frequencies, DDR throughput, system
> > load, etc.
> >
> > My take on this is that the decoder driver should advertise the
> > highest resolution the decoder can handle due to hardware constraints.
> > Performance related things depend on the integration details and
> > should be managed elsewhere. For example Android and Chrome OS manage
> > expected decoding performance in per-board configuration files.
>
> When you read datasheet, the HW is always rated to maximum level (and
> it's a range) with the assumption of a single stream. It seems much
> easier to expose this as-is, statically then to start doing some math
> with data that isn't fully exposed to the user. This is about filtering
> of multiple CODEC instances, it does not need to be rocket science,
> specially that the amount of missing data is important (e.g. usage of
> tiles, compression, IPP all have an impact on the final performance).
> All we want to know in userspace is if this HW is even possibly capable
> of LEVEL X, and if not, we'll look for another one.
>

Agreed, one could potentially define it this way, but would it be
really useful for the userspace and the users? I guess it could enable
slightly faster fallback to software decoding in the extreme case of
the hardware not supporting the level at all, but I suspect that the
majority of cases would be the hardware just being unusably slow.

Also, as I mentioned before, we already return the range of supported
resolutions, which in practice should map to the part of the level
that may depend on hardware capabilities rather than performance, so
exposing levels as well would add redundancy to the information
exposed.

> >
> > > > Signed-off-by: Hirokazu Honda <hiroh@xxxxxxxxxxxx>
> > > > ---
> > > >  drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
> > > > index 6d9d41170832..9387619235d8 100644
> > > > --- a/drivers/staging/media/hantro/hantro_drv.c
> > > > +++ b/drivers/staging/media/hantro/hantro_drv.c
> > > > @@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = {
> > > >                       .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B,
> > > >                       .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B,
> > > >               },
> > > > +     }, {
> > > > +             .codec = HANTRO_H264_DECODER,
> > > > +             .cfg = {
> > > > +                     .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > > > +                     .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
> > > > +                     .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > > > +                     .menu_skip_mask =
> > > > +                     BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
> > > > +                     .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> > > > +             }
> > > >       }, {
> > > >       },
> > > >  };