Re: [PATCH RFC 1/2] media: docs: Clarify V4L2_FMT_FLAG_DYN_RESOLUTION usage

[Hans Verkuil]

On 10/06/2026 05:36, Ming Qian(OSS) wrote:
> On Wed, Jun 10, 2026 at 11:49:06AM +0900, Ming Qian(OSS) wrote:
>> On Tue, Jun 09, 2026 at 02:00:02PM -0400, Nicolas Dufresne wrote:
>>
>> Hi Nicolas,
> 
>>> Hi Ming,
>>>
>>> Le lundi 08 juin 2026 à 06:26 +0000, Ming Qian (OSS) a écrit :
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: sashiko-bot@xxxxxxxxxx <sashiko-bot@xxxxxxxxxx>
>>>>> Sent: Monday, June 8, 2026 12:06 PM
>>>>> To: Ming Qian (OSS) <ming.qian@xxxxxxxxxxx>
>>>>> Cc: imx@xxxxxxxxxxxxxxx; Frank.Li@xxxxxxxxxx; linux-media@xxxxxxxxxxxxxxx
>>>>> Subject: Re: [PATCH RFC 1/2] media: docs: Clarify
>>>>> V4L2_FMT_FLAG_DYN_RESOLUTION usage
>>>>>
>>>>> Thank you for your contribution! Sashiko AI review found 1 potential
>>>>> issue(s)
>>>>> to consider:
>>>>> - [High] The added documentation conflates
>>>>> `V4L2_FMT_FLAG_DYN_RESOLUTION` (which designates mid-stream dynamic
>>>>> resolution switching capability) with the mandatory initial
>>>>> `V4L2_EVENT_SOURCE_CHANGE` event. By stating "Clients can use this flag to
>>>>> determine whether the decoder will signal the initial resolution", it
>>>>> explicitly
>>>>> legitimizes a flawed userspace heuristic that violates the V4L2 stateful
>>>>> decoder
>>>>> specification. The specification mandates that for coded formats containing
>>>>> resolution info, the decoder MUST parse the stream and emit an initial
>>>>> `V4L2_EVENT_SOURCE_CHANGE` event (Initialization Step 4). Compliant drivers
>>>>> (e.g., `coda`) correctly emit this initial event but do NOT set
>>>>> `V4L2_FMT_FLAG_DYN_RESOLUTION` because they lack mid-stream switching
>>>>> capabilities. If userspace follows this documented advice, it will
>>>>> erroneously
>>>>> skip waiting for the initial event on these drivers, breaking Capture Setup
>>>>> and
>>>>> stream decoding.
>>>>> --
>>>>>
>>>>> commit ec9c0583334c246f7813132b55724e42622e8edd
>>>>> Author: Ming Qian <ming.qian@xxxxxxxxxxx>
>>>>>
>>>>> media: docs: Clarify V4L2_FMT_FLAG_DYN_RESOLUTION usage
>>>>>
>>>>> This commit updates the V4L2 stateful decoder documentation to clarify the
>>>>> usage of the V4L2_FMT_FLAG_DYN_RESOLUTION flag. It adds a note stating
>>>>> that decoders setting this flag will always emit an initial source change
>>>>> event.
>>>>> The documentation suggests clients can use this flag to decide whether to
>>>>> wait
>>>>> for the initial resolution signal.
>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>>>> b/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>>>> --- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>>>> @@ -361,6 +361,14 @@ Initialization
>>>>> [ ... ]
>>>>>> +    .. note::
>>>>>> +
>>>>>> +       Decoders that set the ``V4L2_FMT_FLAG_DYN_RESOLUTION`` flag for
>>>>> the
>>>>>> +       coded format will always emit an initial
>>>>> ``V4L2_EVENT_SOURCE_CHANGE``
>>>>>> +       event with ``V4L2_EVENT_SRC_CH_RESOLUTION`` during this phase.
>>>>> Clients
>>>>>> +       can use this flag to determine whether the decoder will signal the
>>>>>> +       initial resolution before producing decoded frames.
>>>>>
>>>>> [Severity: High]
>>>>> Does this guidance conflate the V4L2_FMT_FLAG_DYN_RESOLUTION flag,
>>>>> which designates mid-stream dynamic resolution switching, with the
>>>>> mandatory initial V4L2_EVENT_SOURCE_CHANGE event?
>>>>>
>>>>> The V4L2 Stateful Decoder API requires an initial
>>>>> V4L2_EVENT_SOURCE_CHANGE event during the Initialization phase for stream
>>>>> parsing, irrespective of dynamic resolution support.
>>>>>
>>>>> Compliant drivers like coda correctly emit this initial event but do not set
>>>>> V4L2_FMT_FLAG_DYN_RESOLUTION because they lack mid-stream switching
>>>>> capabilities.
>>>>>
>>>>> If userspace follows this documented advice, wouldn't it erroneously skip
>>>>> waiting for the initial event on these drivers, potentially breaking capture
>>>>> setup
>>>>> and stream decoding?
>>>>>
>>>>> --
>>>>> Sashiko AI review · https://sashiko.dev/#/patchset/20260608-flag-dyn-
>>>>> resolution-v1-0-44dc65516148@xxxxxxxxxxx?part=1
>>>>
>>>> Hi,
>>>>
>>>> Thanks for the review.
>>>>
>>>> You are right that the V4L2 stateful decoder specification states the initial
>>>> V4L2_EVENT_SOURCE_CHANGE is mandatory for coded formats that contain
>>>> resolution information in the stream (Initialization Step 4).
>>>
>>> Be aware that Sashiko is an AI bot, llm words things with extreme conviction,
>>> and it this case forget about backward compatibility from pre-spec.
>>>
>>>>
>>>> However, in practice, GStreamer's v4l2 stateful decoder implementation uses
>>>> V4L2_FMT_FLAG_DYN_RESOLUTION to determine whether to subscribe and wait for
>>>> the initial source change event. The reasoning from the GStreamer side, as
>>>> Nicolas explained [1]:
>>>>
>>>>   
>>>> "
>>>> https://docs.kernel.org/userspace-api/media/v4l/dev-decoder.html#dynamic-resolu
>>>> tion-change
>>>>     Says:
>>>>         Not all decoders can detect resolution changes. Those that do set the
>>>>         V4L2_FMT_FLAG_DYN_RESOLUTION flag.
>>>>
>>>>     So normally that wording should prevent requiring an initial SRC_CH,
>>>>     or emitting later SRC_CH. Your driver don't have this flag, then your
>>>>     driver can't emit this event. But a measure we should take into
>>>>     GStreamer would be to not register (or mark) this event."
>>>>
>>>> @Nicolas, could you elaborate on why GStreamer needs
>>>> V4L2_FMT_FLAG_DYN_RESOLUTION to handle the initial source change event?
>>>> Is this something that should be fixed on the GStreamer side (i.e., always
>>>> wait for the initial event), or is the current heuristic intentional due to
>>>> legacy drivers that don't emit the event?
>>>
>>> The coda source_change notification is completely fake. It does not dependent on
>>> the bitstream content. So the event is left there, since its kind of part of the
>>> ABI, but it does not behave like other implementation, or pre-spec drivers.
>>>
>>> So what we do in GStreamer, is that for legacy driver (no
>>> V4L2_FMT_FLAG_DYN_RESOLUTION), we pre-allocate both queues, based on our guessed
>>> dimensions. If it happens that the conformance windows is small enough, it often
>>> works. DRC will only work if the display dimension changes.
>>>
>>> For any modern driver, that implement V4L2_FMT_FLAG_DYN_RESOLUTION, we strictly
>>> wait for the event, and on DRC, even if the display resolution changes, we let
>>> the driver tell us when to actually reconfigure. The legacy method was kept to
>>> not break coda and older driver, the new method is a lot more reliable, and
>>> avoid allocating twice the capture queue (wrong guess).
>>>
>>> The userspace implementation is also a bit more flexible, as normally the legacy
>>> way should kind of work for any drivers, and we still subscribe it seems. But
>>> the implication is just strange and shouldn't be needed in drivers with
>>> V4L2_FMT_FLAG_DYN_RESOLUTION support.
>>>
>>> Nicolas
>>>
>>>
>>
>> Thanks for the detailed explanation of GStreamer's approach.
>>
>> I have a couple of follow-up questions:
>>
>> 1. Regarding coda's source change being "completely fake":
>>
>>    Looking at the coda driver code, its seq_init_work does parse the
>>    bitstream via hardware (SEQ_INIT command), and the source change event
>>    is only emitted after ctx->initialized is set — which requires the
>>    hardware to successfully parse the stream headers. After the event,
>>    userspace can call G_SELECTION to retrieve the actual display crop
>>    rectangle parsed from the bitstream.
>>
>>    The limitation is that coda requires userspace to set a sufficiently
>>    large resolution via S_FMT(OUTPUT) beforehand (since it validates
>>    stream dimensions fit within the pre-configured buffer size rather
>>    than updating G_FMT with parsed dimensions). But the event itself
>>    does depend on bitstream content and carries useful information
>>    (visible resolution via selection API).
>>
>>    So it seems coda could work with the standard init flow — the source
>>    change event is real, just the information delivery is partial (crop
>>    via G_SELECTION rather than full coded resolution via G_FMT). Would
>>    you agree, or is there another reason GStreamer treats it as legacy?
>>
>> 2. Regarding s5p-mfc:
>>
>>    Interestingly, s5p-mfc sets V4L2_FMT_FLAG_DYN_RESOLUTION but does
>>    NOT emit an initial source change event. After SEQ_DONE, it simply
>>    transitions to MFCINST_HEAD_PARSED state and wakes up waiters —
>>    userspace discovers the resolution by calling G_FMT(CAPTURE) which
>>    internally blocks until header parsing completes.
>>
>>    The source change event is only emitted during mid-stream resolution
>>    changes (RES_CHANGE_FLUSH path). How does GStreamer handle this case?
>>    Does it timeout waiting for the initial event and fall back, or does
>>    it use some other mechanism?
>>
>> Overall, I agree that using V4L2_FMT_FLAG_DYN_RESOLUTION to unify the
>> behavior (both initial source change and mid-stream DRC) is the right
>> direction. But the current state has some inconsistencies:
>>
>>   - coda: emits initial source change, but does NOT set DYN_RESOLUTION
>>   - s5p-mfc: sets DYN_RESOLUTION, but does NOT emit initial source change
>>
>> If we want to document that "DYN_RESOLUTION implies initial source change
>> event will be emitted", s5p-mfc would need to be fixed to comply. Does
>> that seem reasonable, or should we take a different approach?
> 
> Hi,
> 
> I'd like to follow up with a correction regarding the s5p-mfc driver.
> 
> I apologize for the confusion in my earlier analysis. The s5p-mfc driver
> does define V4L2_FMT_FLAG_DYN_RESOLUTION in its internal formats[]
> table. However, the vidioc_enum_fmt() implementation never copies these
> flags to userspace:
> 
>   f->pixelformat = formats[i].fourcc;
>   return 0;  /* f->flags is never set! */
> 
> This is a bug - from userspace's perspective, VIDIOC_ENUM_FMT always
> returns flags = 0, which is why GStreamer treats s5p-mfc via the legacy
> path.
> 
> Looking at the driver's actual behavior:
> 
> - Initial resolution: After header parsing, the driver wakes up the
>   waiting context but does not send V4L2_EVENT_SOURCE_CHANGE. Userspace
>   discovers the resolution by calling G_FMT after STREAMON.
> 
> - Mid-stream resolution change: The driver does send
>   V4L2_EVENT_SOURCE_CHANGE with V4L2_EVENT_SRC_CH_RESOLUTION.
> 
> So if V4L2_FMT_FLAG_DYN_RESOLUTION is defined to cover both initial and
> mid-stream resolution changes:
> 
> - It should not set the flag - because it doesn't fire the event for
>   initial resolution.
> - It should set the flag - because it supports mid-stream DRC via the
>   event.
> 
> Neither setting nor clearing the flag accurately describes the driver's
> current behavior.
> 
> The more natural fix would be to add V4L2_EVENT_SOURCE_CHANGE for the
> initial resolution case as well, then properly report
> V4L2_FMT_FLAG_DYN_RESOLUTION through VIDIOC_ENUM_FMT. If all existing
> drivers that support mid-stream DRC can similarly be updated to fire the
> event for initial resolution, then perhaps the spec change to split the
> flag isn't strictly necessary - drivers just need to be fixed to comply
> with the current definition.
> 
> I'd appreciate hearing others' thoughts on whether this unified approach
> is practical across all existing drivers.

Just a quick reply to mention that the s5p-mfc driver predates a lot of the
newer uAPIs for stateful codecs. It's why it behaves in a not-quite-standard
way. So there is history here. The resolution change support was added in
commit 0520e4cc3bb9d back in 2014, three years after the driver was first
added to the kernel. The stateful decoder spec as we have today was only
introduced in 2019.

Regards,

	Hans

> 
> Best regards,
> Ming
> 
>>
>> Regards,
>> Ming
>>
>>>
>>>>
>>>> [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/work_items/5126
>>>>
>>>> Best regards,
>>>> Ming
>>
>>
>>
>