Re: [PATCH 1/2] media: docs-rst: Document memory-to-memory video decoder interface

[Tomasz Figa]

On Tue, Aug 7, 2018 at 4:13 PM Hans Verkuil <hverkuil@xxxxxxxxx> wrote:
>
> On 07/26/2018 12:20 PM, Tomasz Figa wrote:
> > Hi Hans,
> >
> > On Wed, Jul 25, 2018 at 8:59 PM Hans Verkuil <hverkuil@xxxxxxxxx> wrote:
> >>> +
> >>> +14. Call :c:func:`VIDIOC_STREAMON` to initiate decoding frames.
> >>> +
> >>> +Decoding
> >>> +========
> >>> +
> >>> +This state is reached after a successful initialization sequence. In this
> >>> +state, client queues and dequeues buffers to both queues via
> >>> +:c:func:`VIDIOC_QBUF` and :c:func:`VIDIOC_DQBUF`, following standard
> >>> +semantics.
> >>> +
> >>> +Both queues operate independently, following standard behavior of V4L2
> >>> +buffer queues and memory-to-memory devices. In addition, the order of
> >>> +decoded frames dequeued from ``CAPTURE`` queue may differ from the order of
> >>> +queuing coded frames to ``OUTPUT`` queue, due to properties of selected
> >>> +coded format, e.g. frame reordering. The client must not assume any direct
> >>> +relationship between ``CAPTURE`` and ``OUTPUT`` buffers, other than
> >>> +reported by :c:type:`v4l2_buffer` ``timestamp`` field.
> >>
> >> Is there a relationship between capture and output buffers w.r.t. the timestamp
> >> field? I am not aware that there is one.
> >
> > I believe the decoder was expected to copy the timestamp of matching
> > OUTPUT buffer to respective CAPTURE buffer. Both s5p-mfc and coda seem
> > to be implementing it this way. I guess it might be a good idea to
> > specify this more explicitly.
>
> What about an output buffer producing multiple capture buffers? Or the case
> where the encoded bitstream of a frame starts at one output buffer and ends
> at another? What happens if you have B frames and the order of the capture
> buffers is different from the output buffers?
>
> In other words, for codecs there is no clear 1-to-1 relationship between an
> output buffer and a capture buffer. And we never defined what the 'copy timestamp'
> behavior should be in that case or if it even makes sense.

You're perfectly right. There is no 1:1 relationship, but it doesn't
prevent copying timestamps. It just makes it possible for multiple
CAPTURE buffers to have the same timestamp or some OUTPUT timestamps
not to be found in any CAPTURE buffer.

Best regards,
Tomasz