Re: [PATCH v7 00/34] i.MX Media Driver

From: Steve Longerbeam
Date: Mon May 29 2017 - 14:12:12 EST


Hi Sakari,


On 05/29/2017 08:36 AM, Sakari Ailus wrote:
Hi Hans,

On Mon, May 29, 2017 at 03:46:08PM +0200, Hans Verkuil wrote:
Hi Steve,

On 05/25/2017 02:29 AM, Steve Longerbeam wrote:
In version 7:

- video-mux: switched to Philipp's latest video-mux driver and updated
bindings docs, that makes use of the mmio-mux framework.

- mmio-mux: includes Philipp's temporary patch that adds mmio-mux support
to video-mux driver, until mux framework is merged.

- mmio-mux: updates to device tree from Philipp that define the i.MX6 mux
devices and modifies the video-mux device to become a consumer of the
video mmio-mux.

- minor updates to Documentation/media/v4l-drivers/imx.rst.

- ov5640: do nothing if entity stream count is greater than 1 in
ov5640_s_stream().

- Previous versions of this driver had not tested the ability to enable
multiple independent streams, for instance enabling multiple output
pads from the imx6-mipi-csi2 subdevice, or enabling both prpenc and
prpvf outputs. Marek Vasut tested this support and reported issues
with it.

v4l2_pipeline_inherit_controls() used the media graph walk APIs, but
that walks both sink and source pads, so if there are multiple paths
enabled to video capture devices, controls would be added to the wrong
video capture device, and no controls added to the other enabled
capture devices.

These issues have been fixed. Control inheritance works correctly now
even with multiple enabled capture paths, and (for example)
simultaneous capture from prpenc and prpvf works also, and each with
independent scaling, CSC, and controls. For example prpenc can be
capturing with a 90 degree rotation, while prpvf is capturing with
vertical flip.

So the v4l2_pipeline_inherit_controls() patch has been dropped. The
new version of control inheritance could be made generically available,
but it would be more involved to incorporate it into v4l2-core.

- A new function imx_media_fill_default_mbus_fields() is added to setup
colorimetry at sink pads, and these are propagated to source pads.

- Ensure that the current sink and source rectangles meet alignment
restrictions before applying a new rotation control setting in
prp-enc/vf subdevices.

- Chain the s_stream() subdev calls instead of implementing a custom
stream on/off function that attempts to call a fixed set of subdevices
in a pipeline in the correct order. This also simplifies imx6-mipi-csi2
subdevice, since the correct MIPI CSI-2 startup sequence can be
enforced completely in s_stream(), and s_power() is no longer
required. This also paves the way for more arbitrary OF graphs
external to the i.MX6.

- Converted the v4l2_subdev and media_entity ops structures to const.
What is the status as of v7?

From what I can tell patch 2/34 needs an Ack from Rob Herring, patches
4-14 are out of scope for the media subsystem, patches 20-25 and 27-34
are all staging (so fine to be merged from my point of view).

I'm not sure if patch 26 (defconfig) should be applied while the imx
driver is in staging. I would suggest that this patch is moved to the end
of the series.

That leaves patches 15-19. I replied to patch 15 with a comment, patches
16-18 look good to me, although patches 17 and 18 should be combined to one
patch since patch 17 won't compile otherwise. Any idea when the multiplexer is
expected to be merged? (just curious)

I would really like to get this merged for 4.13, so did I miss anything?
From what I can tell it is really just an Ack for patch 2/34.
The patchset re-initialises the control handler in the video node in its
media device link_setup callback, among other things. I don't think that
this as such should prevent including the driver in staging, but at the same
time my concern is what kind of guarantees can we give in terms of V4L2
framework support on things like this using the current API. I'd say "none".

Re-initializing a control handler in itself works fine (maybe we should
add a v4l2_ctrl_handler_reinit() ?).

I agree that calling free/init/add at this time (in link_setup and link_notify)
is a new way to use those APIs. The effect is that a video node's controls
can change post driver load.

There's nothing in Documentation/kapi/v4l2-controls.rst that states that
a video node's control list can't change at runtime. But as I said free/init/add
works just fine and can be called any time after a subdevice or video device
is registered.


If there's a need for this (there should not be, as the controls are exposed
to the user space through the sub-device nodes as the other drivers do), the
framework APIs need to be extended.

Right, this gets back to the media framework usability arguments. At least
myself, Philipp, and Russell feel that automatic inheritance of a configured
pipeline's controls to a video device adds to the usability.


I think it'd be good to add a TODO file (such as the one in OMAP4ISS) as
part of the patchset that would include a list of things left to do so
they're not forgotten.

Right, there is a TODO, but I will add an item that mentions this included
feature and whether it should eventually become part of the framework
or dropped.

Steve