Re: [RFC PATCH 12/16] xen/xenbus: support xenbus frontend/backend with xenhost_t

[Ankur Arora]

On 6/17/19 2:50 AM, Juergen Gross wrote:
> On 09.05.19 19:25, Ankur Arora wrote:
> > As part of xenbus init, both frontend, backend interfaces need to talk
> > on the correct xenbus. This might be a local xenstore (backend) or might
> > be a XS_PV/XS_HVM interface (frontend) which needs to talk over xenbus
> > with the remote xenstored. We bootstrap all of these with evtchn/gfn
> > parameters from (*setup_xs)().
> >
> > Given this we can do appropriate device discovery (in case of frontend)
> > and device connectivity for the backend.
> > Once done, we stash the xenhost_t * in xen_bus_type, xenbus_device or
> > xenbus_watch and then the frontend and backend devices implicitly use
> > the correct interface.
> >
> > The rest of patch is just changing the interfaces where needed.
> >
> > Signed-off-by: Ankur Arora <ankur.a.arora@xxxxxxxxxx>
> > ---
> > Â drivers/block/xen-blkback/blkback.cÂÂÂÂÂÂÂ |Â 10 +-
> > Â drivers/net/xen-netfront.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 14 +-
> > Â drivers/pci/xen-pcifront.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 4 +-
> > Â drivers/xen/cpu_hotplug.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 4 +-
> > Â drivers/xen/manage.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 28 +--
> > Â drivers/xen/xen-balloon.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 8 +-
> > Â drivers/xen/xenbus/xenbus.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 45 ++--
> > Â drivers/xen/xenbus/xenbus_client.cÂÂÂÂÂÂÂÂ |Â 32 +--
> > Â drivers/xen/xenbus/xenbus_comms.cÂÂÂÂÂÂÂÂÂ | 121 +++++-----
> > Â drivers/xen/xenbus/xenbus_dev_backend.cÂÂÂ |Â 30 ++-
> > Â drivers/xen/xenbus/xenbus_dev_frontend.cÂÂ |Â 22 +-
> > Â drivers/xen/xenbus/xenbus_probe.cÂÂÂÂÂÂÂÂÂ | 246 +++++++++++++--------
> >  drivers/xen/xenbus/xenbus_probe_backend.c | 19 +-
> > Â drivers/xen/xenbus/xenbus_probe_frontend.c |Â 65 +++---
> > Â drivers/xen/xenbus/xenbus_xs.cÂÂÂÂÂÂÂÂÂÂÂÂ | 188 +++++++++-------
> > Â include/xen/xen-ops.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 3 +
> > Â include/xen/xenbus.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 54 +++--
> > Â include/xen/xenhost.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 20 ++
> > Â 18 files changed, 536 insertions(+), 377 deletions(-)
> >
> > diff --git a/drivers/xen/xenbus/xenbus_dev_frontend.c 
> > b/drivers/xen/xenbus/xenbus_dev_frontend.c
> > index c3e201025ef0..d6e0c397c6a0 100644
> > --- a/drivers/xen/xenbus/xenbus_dev_frontend.c
> > +++ b/drivers/xen/xenbus/xenbus_dev_frontend.c
> > @@ -58,10 +58,14 @@
> > Â #include <xen/xenbus.h>
> > Â #include <xen/xen.h>
> > +#include <xen/interface/xen.h>
> > +#include <xen/xenhost.h>
> > Â #include <asm/xen/hypervisor.h>
> > Â #include "xenbus.h"
> > +static xenhost_t *xh;
> > +
> > Â /*
> > ÂÂ * An element of a list of outstanding transactions, for which we're
> > ÂÂ * still waiting a reply.
> > @@ -312,13 +316,13 @@ static void xenbus_file_free(struct kref *kref)
> > ÂÂÂÂÂÂ */
> > ÂÂÂÂÂ list_for_each_entry_safe(trans, tmp, &u->transactions, list) {
> > -ÂÂÂÂÂÂÂ xenbus_transaction_end(trans->handle, 1);
> > +ÂÂÂÂÂÂÂ xenbus_transaction_end(xh, trans->handle, 1);
> > ÂÂÂÂÂÂÂÂÂ list_del(&trans->list);
> > ÂÂÂÂÂÂÂÂÂ kfree(trans);
> > ÂÂÂÂÂ }
> > ÂÂÂÂÂ list_for_each_entry_safe(watch, tmp_watch, &u->watches, list) {
> > -ÂÂÂÂÂÂÂ unregister_xenbus_watch(&watch->watch);
> > +ÂÂÂÂÂÂÂ unregister_xenbus_watch(xh, &watch->watch);
> > ÂÂÂÂÂÂÂÂÂ list_del(&watch->list);
> > ÂÂÂÂÂÂÂÂÂ free_watch_adapter(watch);
> > ÂÂÂÂÂ }
> > @@ -450,7 +454,7 @@ static int xenbus_write_transaction(unsigned 
> > msg_type,
> > ÂÂÂÂÂÂÂÂÂÂÂÂ (!strcmp(msg->body, "T") || !strcmp(msg->body, "F"))))
> > ÂÂÂÂÂÂÂÂÂ return xenbus_command_reply(u, XS_ERROR, "EINVAL");
> > -ÂÂÂ rc = xenbus_dev_request_and_reply(&msg->hdr, u);
> > +ÂÂÂ rc = xenbus_dev_request_and_reply(xh, &msg->hdr, u);
> > ÂÂÂÂÂ if (rc && trans) {
> > ÂÂÂÂÂÂÂÂÂ list_del(&trans->list);
> > ÂÂÂÂÂÂÂÂÂ kfree(trans);
> > @@ -489,7 +493,7 @@ static int xenbus_write_watch(unsigned msg_type, 
> > struct xenbus_file_priv *u)
> > ÂÂÂÂÂÂÂÂÂ watch->watch.callback = watch_fired;
> > ÂÂÂÂÂÂÂÂÂ watch->dev_data = u;
> > -ÂÂÂÂÂÂÂ err = register_xenbus_watch(&watch->watch);
> > +ÂÂÂÂÂÂÂ err = register_xenbus_watch(xh, &watch->watch);
> > ÂÂÂÂÂÂÂÂÂ if (err) {
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂ free_watch_adapter(watch);
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂ rc = err;
> > @@ -500,7 +504,7 @@ static int xenbus_write_watch(unsigned msg_type, 
> > struct xenbus_file_priv *u)
> > ÂÂÂÂÂÂÂÂÂ list_for_each_entry(watch, &u->watches, list) {
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂ if (!strcmp(watch->token, token) &&
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ !strcmp(watch->watch.node, path)) {
> > -ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unregister_xenbus_watch(&watch->watch);
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unregister_xenbus_watch(xh, &watch->watch);
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ list_del(&watch->list);
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ free_watch_adapter(watch);
> > ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ break;
> > @@ -618,8 +622,9 @@ static ssize_t xenbus_file_write(struct file *filp,
> > Â static int xenbus_file_open(struct inode *inode, struct file *filp)
> > Â {
> > ÂÂÂÂÂ struct xenbus_file_priv *u;
> > +ÂÂÂ struct xenstore_private *xs = xs_priv(xh);
> > -ÂÂÂ if (xen_store_evtchn == 0)
> > +ÂÂÂ if (xs->store_evtchn == 0)
> > ÂÂÂÂÂÂÂÂÂ return -ENOENT;
> > ÂÂÂÂÂ nonseekable_open(inode, filp);
> > @@ -687,6 +692,11 @@ static int __init xenbus_init(void)
> > ÂÂÂÂÂ if (!xen_domain())
> > ÂÂÂÂÂÂÂÂÂ return -ENODEV;
> > +ÂÂÂ if (xen_driver_domain() && xen_nested())
> > +ÂÂÂÂÂÂÂ xh = xh_remote;
> > +ÂÂÂ else
> > +ÂÂÂÂÂÂÂ xh = xh_default;
>
> This precludes any mixed use of L0 and L1 frontends. With this move you
> make it impossible to e.g. use a driver domain for networking in L1 with
> a L1-local PV disk, or pygrub in L1 dom0.
Ah, yes. I hadn't thought about that case.

Let me see how I can rework this interface.

Ankur

> Juergen