Re: [PATCH RESEND v2 1/2] drm/xen-front: Add support for Xen PV display frontend

From: Oleksandr Andrushchenko
Date: Tue Mar 20 2018 - 07:58:13 EST

On 03/19/2018 05:28 PM, Daniel Vetter wrote:
On Mon, Mar 19, 2018 at 3:19 PM, Oleksandr Andrushchenko
<andr2000@xxxxxxxxx> wrote:
On 03/19/2018 03:51 PM, Daniel Vetter wrote:
On Fri, Mar 16, 2018 at 12:52:09PM +0200, Oleksandr Andrushchenko wrote:
Hi, Daniel!
Sorry, if I strip the patch too much below.

On 03/16/2018 10:23 AM, Daniel Vetter wrote:
S-o-b line went missing here :-)
will restore it back ;)
I've read through it, 2 actual review comments (around hot-unplug and
around the error recovery for failed flips), a few bikesheds, but looks
all reasonable to me. And much easier to read as one big patch (it's
just
3k).

One more thing I'd do as a follow-up (don't rewrite everything, this is
close to merge, better to get it in first): You have a lot of
indirections
and function calls across sources files. That's kinda ok if you have a
huge driver with 100+k lines of code where you have to split things up.
But for a small driver like yours here it's a bit overkill.
will review and try to rework after the driver is in
I'll probably merge xen_drm_front_drv.c and xen_drm_front.c now as
anyway I have to re-work driver unloading, e.g. "fishy" code below.
Personally I'd merge at least the xen backend stuff into the
corresponding
kms code, but that's up to you.
I prefer to have it in smaller chunks and all related code at
one place, so it is easier to maintain. That is why I didn't
plumb frontend <-> backend code right into the KMS code.
And as mentioned, if you decide to do
that, a follow-up patch (once this has merged) is perfectly fine.
Ok, after the merge
If you prefer your current layout, then pls keep it. Bikeshed = personal
style nit, feel free to ignore if you like stuff differently. In the end
it's your driver, not mine, and I can easily navigate the current code
(with a few extra jumps).
Some of the indirections will be removed by merging
xen_drm_front_drv.c and xen_drm_front.c. Are these what you
mean or is there anything else?

-Daniel

-Daniel

+int xen_drm_front_dbuf_create_from_sgt(struct xen_drm_front_info
*front_info,
+ uint64_t dbuf_cookie, uint32_t width, uint32_t height,
+ uint32_t bpp, uint64_t size, struct sg_table *sgt)
+{
+ return be_dbuf_create_int(front_info, dbuf_cookie, width,
height,
+ bpp, size, NULL, sgt);
+}
+
+int xen_drm_front_dbuf_create_from_pages(struct xen_drm_front_info
*front_info,
+ uint64_t dbuf_cookie, uint32_t width, uint32_t height,
+ uint32_t bpp, uint64_t size, struct page **pages)
+{
+ return be_dbuf_create_int(front_info, dbuf_cookie, width,
height,
+ bpp, size, pages, NULL);
+}
The above two wrappers seem a bit much, just to set sgt = NULL or pages
=
NULL in one of them. I'd drop them, but that's a bikeshed so feel free
to
ignore.
I had that the way you say in some of the previous implementations,
but finally decided to have these dummy wrappers: seems
to be cleaner this way
+static void displback_disconnect(struct xen_drm_front_info
*front_info)
+{
+ bool removed = true;
+
+ if (front_info->drm_pdev) {
+ if (xen_drm_front_drv_is_used(front_info->drm_pdev)) {
+ DRM_WARN("DRM driver still in use, deferring
removal\n");
+ removed = false;
+ } else
+ xen_drv_remove_internal(front_info);
Ok this logic here is fishy, since you're open-coding the drm unplug
infrastructure, but slightly differently and slightyl racy. If you have
a
driver where your underlying "hw" (well it's virtual here, but same
idea)
can disappear any time while userspace is still using the drm driver,
you
need to use the drm_dev_unplug() function and related code.
drm_dev_unplug() works like drm_dev_unregister, except for the hotplug
case.

Then you also have to guard all the driver entry points where you do
access the backchannel using drm_dev_is_unplugged() (I've seen a few of
those already). Then you can rip out all the logic here and the
xen_drm_front_drv_is_used() helper.
Will rework it with drm_dev_unplug, thank you
I thought there's some patches from Noralf in-flight that improved the
docs on this, I need to check
Yes, I will definitely use those as soon as they are available.
But at the moment let me clarify a bit on the use-cases for driver
unplugging and backend disconnection.

The backend, by disconnecting, expects full DRM driver teardown, because,
for example, it might need to replace current frontendâs configuration
completely or stop supporting para-virtualized display for some reason.

This means that once I have displback_disconnected callback (on XenBus state
change) I am trying to unregister and remove the DRM driver which seems to
be
not possible if I have relevant code in DRM callbacks (e.g. I cannot try
removing
driver from driver's callback).

So, even if I add drm_dev_unplug (which anyway seems to be the right thing)
Iâll have to have that fishy code for XenBus state handling.

These are the unplug/disconnect use-cases we have:

1. Rmmod
========
1.1. If DRM driver is not in use
We can call xen_drv_remove_internal immediately and remove both DRM and
XenBus drivers

1.2. If DRM driver is in use
In this case usage count of the module is non-zero and driver cannot be
removed

2. Backend disconnect callback
==============================
2.0. Call drm_dev_unplug as the first step

2.1. If DRM driver is not in use (dev->open_count == 0)
(The check of dev->open_count against zero should be safe,
as before that we call drm_dev_unplug, so the framework will not allow
open_count
to be incremented).
This is similar to 1.1 and we can call xen_drv_remove_internal immediately
and
remove both DRM and XenBus drivers

2.2. DRM driver is in use (dev->open_count != 0)
This seems to be the only *really fishy place*.
In this case drm_dev_unplug will not allow new clients for the DRM driver,
but we cannot start removing DRM driver until the last client closes the DRM
device.

This is the change I am planning to introduce:

User-space may hold the DRM device in use for unlimited time, so we cannot
hang in XenBusâ
displback_disconnected callback indefinitely, but have to defer DRM driver
deletion:
we switch frontend driverâs XenBus state to XenbusStateReconfiguring
(telling the backend to
wait until we release the DRM driver) and on drm_drv.lastclose callback
schedule a deferred
work which will check for dev->open_count == 0 to become true, so we can
remove DRM driver
and change frontendâs XenBus state to XenbusStateInitialising.
If at the time of the check dev->open_count != 0 then we reschedule the same
work with
start up delay, and re-check later. This rescheduling happens until
dev->open_count == 0,
which is the marker for DRM driver removal.

Does the above makes sense?
If so, then Iâll have this implemented in v4 of the driver.
There should be no difference between immediate removal and delayed
removal of the drm_device from the xenbus pov. The lifetimes of the
front-end (drm_device) and backend (the xen bus thing) are entirely
decoupled:
Well, they are not decoupled for simplicity of handling,
please see below

So for case 2 you only have 1 case:

- drm_dev_unplug
- tear down the entire xenbus backend completely
- all xenbus access will be caught with drm_dev_entre/exit (well right
now drm_dev_is_unplugged) checks, including any access to your private
drm_device data
- once drm_device->open_count == 0 the core will tear down the
drm_device instance and call your optional drm_driver->release
callback.

So past drm_dev_unplug the drm_device is in zombie state and the only
thing that will happen is a) it rejects all ioctls and anything else
userspace might ask it to do and b) gets releases once the last
userspace reference is gone.
I have re-worked the driver with this in mind [1]
So, I now use drm_dev_unplug and destroy the DRM device
on drm_driver.release.
In context of unplug work I also merged xen_drm_front_drv.c and
xen_drm_front.c as these are too coupled together now.

Could you please take a look and tell me if this is what you mean?

If the backend comes up again, you create a _new_ drm_device instance
(while the other one is still in the process of eventually getting
released).
We only have a single xenbus instance, so this way I'll need
to handle list of such zombies. For that reason I prefer to
wait until the DRM device is destroyed, telling the backend
to hold on until then (via going into XenbusStateReconfiguring state).

Another drawback of such approach is that I'll have different
minors at run-time, e.g. card0, card1, etc.
For software which has /dev/dri/card0 hardcoded it may be a problem.
But this is minor, IMO

In short, your driver code should never have a need to look at
drm_device->open_count. I hope this explains it a bit better.
-Daniel

Yes, you are correct: at [1] I am not touching drm_device->open_count
anymore and everything just happens synchronously

+#define XEN_DRM_NUM_VIDEO_MODES 1
+#define XEN_DRM_CRTC_VREFRESH_HZ 60
+
+static int connector_get_modes(struct drm_connector *connector)
+{
+ struct xen_drm_front_drm_pipeline *pipeline =
+ to_xen_drm_pipeline(connector);
+ struct drm_display_mode *mode;
+ struct videomode videomode;
+ int width, height;
+
+ mode = drm_mode_create(connector->dev);
+ if (!mode)
+ return 0;
+
+ memset(&videomode, 0, sizeof(videomode));
+ videomode.hactive = pipeline->width;
+ videomode.vactive = pipeline->height;
+ width = videomode.hactive + videomode.hfront_porch +
+ videomode.hback_porch + videomode.hsync_len;
+ height = videomode.vactive + videomode.vfront_porch +
+ videomode.vback_porch + videomode.vsync_len;
+ videomode.pixelclock = width * height *
XEN_DRM_CRTC_VREFRESH_HZ;
+ mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER;
+
+ drm_display_mode_from_videomode(&videomode, mode);
+ drm_mode_probed_add(connector, mode);
+ return XEN_DRM_NUM_VIDEO_MODES;
Bikeshed: just hardcode this to 1, the #define is imo more confusing.
ok, will remove #define
+
+ }
+ /*
+ * Send page flip request to the backend *after* we have event
cached
+ * above, so on page flip done event from the backend we can
+ * deliver it and there is no race condition between this code
and
+ * event from the backend.
+ * If this is not a page flip, e.g. no flip done event from the
backend
+ * is expected, then send now.
+ */
+ if (!display_send_page_flip(pipe, old_plane_state))
+ xen_drm_front_kms_send_pending_event(pipeline);
The control flow here is a bit confusing. I'd put the call to send out
the
event right away in case of a failure to communicate with the backend
into
display_send_page_flip() itself. Then drop the bool return value and
make
it void, and also push the comment explaining what you do in case of
errors into that function.
The reason for having bool for page flip here is that we
need to send pending event for display enable/disable, for example.
So, I decided to make it this way:
1. page flip handled - handles pending event internally
(defers sending until frame done event from the backend)
2. page flip failed - handles externally in case of any
page flip related error, e.g. "not handled" cases, either
due to backend communication error or whatever else
3. all other cases, but page flip
That way the error handling and recovery is all neatly tied together in
one place instead of spread around.
Well, I tried to keep it all at one place, but as we decided
to implement connector hotplug for error delivery it
became split. Also, I handle frame done event time-outs there.
You can leave things as-is if you prefer, just for me it looked a bit
confusion and unecessarily complex.
I'll think more if I can simplify this
-Daniel
Thank you,
Oleksandr Andrushchenko

_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/dri-devel


Thank you very much for your time and comments,
Oleksandr

[1] https://github.com/andr2000/linux/commits/drm_tip_pv_drm_v3