Re: [PATCH RFC 1/2] media: docs: Clarify V4L2_FMT_FLAG_DYN_RESOLUTION usage

[Nicolas Dufresne]

Le mercredi 10 juin 2026 à 11:49 +0900, Ming Qian(OSS) a écrit :
> On Tue, Jun 09, 2026 at 02:00:02PM -0400, Nicolas Dufresne wrote:
> 
> Hi Nicolas,
> > Hi Ming,
> > 
> > Le lundi 08 juin 2026 à 06:26 +0000, Ming Qian (OSS) a écrit :
> > > 
> > > 
> > > > -----Original Message-----
> > > > From: sashiko-bot@xxxxxxxxxx <sashiko-bot@xxxxxxxxxx>
> > > > Sent: Monday, June 8, 2026 12:06 PM
> > > > To: Ming Qian (OSS) <ming.qian@xxxxxxxxxxx>
> > > > Cc: imx@xxxxxxxxxxxxxxx; Frank.Li@xxxxxxxxxx; linux-media@xxxxxxxxxxxxxxx
> > > > Subject: Re: [PATCH RFC 1/2] media: docs: Clarify
> > > > V4L2_FMT_FLAG_DYN_RESOLUTION usage
> > > > 
> > > > Thank you for your contribution! Sashiko AI review found 1 potential
> > > > issue(s)
> > > > to consider:
> > > > - [High] The added documentation conflates
> > > > `V4L2_FMT_FLAG_DYN_RESOLUTION` (which designates mid-stream dynamic
> > > > resolution switching capability) with the mandatory initial
> > > > `V4L2_EVENT_SOURCE_CHANGE` event. By stating "Clients can use this flag to
> > > > determine whether the decoder will signal the initial resolution", it
> > > > explicitly
> > > > legitimizes a flawed userspace heuristic that violates the V4L2 stateful
> > > > decoder
> > > > specification. The specification mandates that for coded formats containing
> > > > resolution info, the decoder MUST parse the stream and emit an initial
> > > > `V4L2_EVENT_SOURCE_CHANGE` event (Initialization Step 4). Compliant drivers
> > > > (e.g., `coda`) correctly emit this initial event but do NOT set
> > > > `V4L2_FMT_FLAG_DYN_RESOLUTION` because they lack mid-stream switching
> > > > capabilities. If userspace follows this documented advice, it will
> > > > erroneously
> > > > skip waiting for the initial event on these drivers, breaking Capture Setup
> > > > and
> > > > stream decoding.
> > > > --
> > > > 
> > > > commit ec9c0583334c246f7813132b55724e42622e8edd
> > > > Author: Ming Qian <ming.qian@xxxxxxxxxxx>
> > > > 
> > > > media: docs: Clarify V4L2_FMT_FLAG_DYN_RESOLUTION usage
> > > > 
> > > > This commit updates the V4L2 stateful decoder documentation to clarify the
> > > > usage of the V4L2_FMT_FLAG_DYN_RESOLUTION flag. It adds a note stating
> > > > that decoders setting this flag will always emit an initial source change
> > > > event.
> > > > The documentation suggests clients can use this flag to decide whether to
> > > > wait
> > > > for the initial resolution signal.
> > > > 
> > > > > diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > > > b/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > > > --- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > > > @@ -361,6 +361,14 @@ Initialization
> > > > [ ... ]
> > > > > +    .. note::
> > > > > +
> > > > > +       Decoders that set the ``V4L2_FMT_FLAG_DYN_RESOLUTION`` flag for
> > > > the
> > > > > +       coded format will always emit an initial
> > > > ``V4L2_EVENT_SOURCE_CHANGE``
> > > > > +       event with ``V4L2_EVENT_SRC_CH_RESOLUTION`` during this phase.
> > > > Clients
> > > > > +       can use this flag to determine whether the decoder will signal the
> > > > > +       initial resolution before producing decoded frames.
> > > > 
> > > > [Severity: High]
> > > > Does this guidance conflate the V4L2_FMT_FLAG_DYN_RESOLUTION flag,
> > > > which designates mid-stream dynamic resolution switching, with the
> > > > mandatory initial V4L2_EVENT_SOURCE_CHANGE event?
> > > > 
> > > > The V4L2 Stateful Decoder API requires an initial
> > > > V4L2_EVENT_SOURCE_CHANGE event during the Initialization phase for stream
> > > > parsing, irrespective of dynamic resolution support.
> > > > 
> > > > Compliant drivers like coda correctly emit this initial event but do not set
> > > > V4L2_FMT_FLAG_DYN_RESOLUTION because they lack mid-stream switching
> > > > capabilities.
> > > > 
> > > > If userspace follows this documented advice, wouldn't it erroneously skip
> > > > waiting for the initial event on these drivers, potentially breaking capture
> > > > setup
> > > > and stream decoding?
> > > > 
> > > > --
> > > > Sashiko AI review · https://sashiko.dev/#/patchset/20260608-flag-dyn-
> > > > resolution-v1-0-44dc65516148@xxxxxxxxxxx?part=1
> > > 
> > > Hi,
> > > 
> > > Thanks for the review.
> > > 
> > > You are right that the V4L2 stateful decoder specification states the initial
> > > V4L2_EVENT_SOURCE_CHANGE is mandatory for coded formats that contain
> > > resolution information in the stream (Initialization Step 4).
> > 
> > Be aware that Sashiko is an AI bot, llm words things with extreme conviction,
> > and it this case forget about backward compatibility from pre-spec.
> > 
> > > 
> > > However, in practice, GStreamer's v4l2 stateful decoder implementation uses
> > > V4L2_FMT_FLAG_DYN_RESOLUTION to determine whether to subscribe and wait for
> > > the initial source change event. The reasoning from the GStreamer side, as
> > > Nicolas explained [1]:
> > > 
> > >   
> > > "
> > > https://docs.kernel.org/userspace-api/media/v4l/dev-decoder.html#dynamic-resolu
> > > tion-change
> > >     Says:
> > >         Not all decoders can detect resolution changes. Those that do set the
> > >         V4L2_FMT_FLAG_DYN_RESOLUTION flag.
> > > 
> > >     So normally that wording should prevent requiring an initial SRC_CH,
> > >     or emitting later SRC_CH. Your driver don't have this flag, then your
> > >     driver can't emit this event. But a measure we should take into
> > >     GStreamer would be to not register (or mark) this event."
> > > 
> > > @Nicolas, could you elaborate on why GStreamer needs
> > > V4L2_FMT_FLAG_DYN_RESOLUTION to handle the initial source change event?
> > > Is this something that should be fixed on the GStreamer side (i.e., always
> > > wait for the initial event), or is the current heuristic intentional due to
> > > legacy drivers that don't emit the event?
> > 
> > The coda source_change notification is completely fake. It does not dependent on
> > the bitstream content. So the event is left there, since its kind of part of the
> > ABI, but it does not behave like other implementation, or pre-spec drivers.
> > 
> > So what we do in GStreamer, is that for legacy driver (no
> > V4L2_FMT_FLAG_DYN_RESOLUTION), we pre-allocate both queues, based on our guessed
> > dimensions. If it happens that the conformance windows is small enough, it often
> > works. DRC will only work if the display dimension changes.
> > 
> > For any modern driver, that implement V4L2_FMT_FLAG_DYN_RESOLUTION, we strictly
> > wait for the event, and on DRC, even if the display resolution changes, we let
> > the driver tell us when to actually reconfigure. The legacy method was kept to
> > not break coda and older driver, the new method is a lot more reliable, and
> > avoid allocating twice the capture queue (wrong guess).
> > 
> > The userspace implementation is also a bit more flexible, as normally the legacy
> > way should kind of work for any drivers, and we still subscribe it seems. But
> > the implication is just strange and shouldn't be needed in drivers with
> > V4L2_FMT_FLAG_DYN_RESOLUTION support.
> > 
> > Nicolas
> > 
> > 
> 
> Thanks for the detailed explanation of GStreamer's approach.
> 
> I have a couple of follow-up questions:
> 
> 1. Regarding coda's source change being "completely fake":
> 
>    Looking at the coda driver code, its seq_init_work does parse the
>    bitstream via hardware (SEQ_INIT command), and the source change event
>    is only emitted after ctx->initialized is set — which requires the
>    hardware to successfully parse the stream headers. After the event,
>    userspace can call G_SELECTION to retrieve the actual display crop
>    rectangle parsed from the bitstream.
> 
>    The limitation is that coda requires userspace to set a sufficiently
>    large resolution via S_FMT(OUTPUT) beforehand (since it validates
>    stream dimensions fit within the pre-configured buffer size rather
>    than updating G_FMT with parsed dimensions). But the event itself
>    does depend on bitstream content and carries useful information
>    (visible resolution via selection API).
> 
>    So it seems coda could work with the standard init flow — the source
>    change event is real, just the information delivery is partial (crop
>    via G_SELECTION rather than full coded resolution via G_FMT). Would
>    you agree, or is there another reason GStreamer treats it as legacy?

Its possible I miss-understood the code indeed.

> 
> 2. Regarding s5p-mfc:
> 
>    Interestingly, s5p-mfc sets V4L2_FMT_FLAG_DYN_RESOLUTION but does
>    NOT emit an initial source change event. After SEQ_DONE, it simply
>    transitions to MFCINST_HEAD_PARSED state and wakes up waiters —
>    userspace discovers the resolution by calling G_FMT(CAPTURE) which
>    internally blocks until header parsing completes.
> 
>    The source change event is only emitted during mid-stream resolution
>    changes (RES_CHANGE_FLUSH path). How does GStreamer handle this case?
>    Does it timeout waiting for the initial event and fall back, or does
>    it use some other mechanism?

That one I have no idea, that MFC driver gets highly hacked up on Android, and
Samsung is pretty much gone from the general purpose SBC or IoT world. Last time
I worked with MFC, they did not have that flags set (it didn't exist yet), and
the blocking G_FMT was the thing that pretty much everyone disliked, and why we
made a spec for newly introduce decoder.

I'd assume that GStreamer cannot operate on mainline MFC. It would be nice to
fix, but I don't myself have the bandwidth. Is there anyone left at Samsung that
cares about it ? Adding the two maintainers. Though clearly, its non spec
compliant to flag V4L2_FMT_FLAG_DYN_RESOLUTION without emitting the initial
event.

> 
> Overall, I agree that using V4L2_FMT_FLAG_DYN_RESOLUTION to unify the
> behavior (both initial source change and mid-stream DRC) is the right
> direction. But the current state has some inconsistencies:
> 
>   - coda: emits initial source change, but does NOT set DYN_RESOLUTION
>   - s5p-mfc: sets DYN_RESOLUTION, but does NOT emit initial source change
> 
> If we want to document that "DYN_RESOLUTION implies initial source change
> event will be emitted", s5p-mfc would need to be fixed to comply. Does
> that seem reasonable, or should we take a different approach?

I think though that we don't have to have that documented, since
V4L2_FMT_FLAG_DYN_RESOLUTION comes after the V4L2 stateful spec, and so, if you
include that flag you also have to comply to the spec. Maybe I'm a bit wishful,
but the reality is that this is more about unmaintained territory.

For CODA, its very very old, and the legacy way of guessing the allocation is
what works best there. Though, I'd be happy to help fix and test this one, I do
have this hardware, and surprisingly, imx6 which host this IP is still in use
today.

Nicolas

> 
> Regards,
> Ming
> 
> > 
> > > 
> > > [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/work_items/5126
> > > 
> > > Best regards,
> > > Ming
>

Attachment: signature.asc
Description: This is a digitally signed message part