Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus devices

[Greg KH]

On Tue, Oct 18, 2022 at 07:36:54AM +0000, Long Li wrote:
> > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus
> > devices
> > 
> > On Tue, Oct 18, 2022 at 06:57:33AM +0000, Long Li wrote:
> > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed
> > > > VMBus devices
> > > >
> > > > On Tue, Oct 18, 2022 at 06:31:16AM +0000, Long Li wrote:
> > > > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low
> > > > > > speed VMBus devices
> > > > > >
> > > > > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote:
> > > > > > > Hyper-V is adding some "specialty" synthetic devices.
> > > > > >
> > > > > > What devices are those specifically?
> > > > > >
> > > > > > > Instead of writing new kernel-level VMBus drivers for these
> > > > > > > devices, the devices will be presented to user space via this
> > > > > > > existing Hyper-V generic UIO driver, so that a user space
> > > > > > > driver can
> > > > handle the device.
> > > > > > > Since these new synthetic devices are low speed devices, they
> > > > > > > don't support monitor bits and we must use vmbus_setevent() to
> > > > > > > enable interrupts from the host.
> > > > > >
> > > > > > That is not what the UIO interface is for.  Please write real
> > > > > > drivers so that they tie into the specific user/kernel apis for
> > > > > > those device
> > > > types.
> > > > > >
> > > > > > Without a specific list of what these devices are, I can not
> > > > > > recommend that anyone use the UIO api for them as that's
> > > > > > probably not
> > > > a good idea.
> > > > >
> > > > > There are some VMBUS drivers currently not implemented in Linux.
> > > > > Out of all VMBUS drivers, two use "monitored bits": they are
> > > > > network and
> > > > storage drivers.
> > > > > All the rest VMBUS drivers use hypercall for host notification and
> > > > > signal for next interrupt. One example of such driver is to
> > > > > collect process level crash information for diagnostic purposes.
> > > > >
> > > > > Also, we want to move some existing kernel mode VMBUS drivers to
> > > > > user-space, such as hv_kvp and hv_filecopy. They don't really fit
> > > > > into an existing kernel API, and they create their own devices
> > > > > under /dev and communicates with a user-mode daemon to do most of
> > > > > the work. It's a better model that we can move those drivers entirely
> > into user-mode.
> > > >
> > > > How are you going to be able to remove drivers that export an
> > > > existing user/kernel api and not break current systems?
> > >
> > > It will be some configuration challenges, but it's doable.
> > 
> > How exactly?
> > 
> > We can not break userspace when you upgrade a kernel version.
> > 
> > > Newer Linux
> > > VMs can be configured in a way to use the UIO interfaces. This doesn't
> > > break any existing VMs because the old model will continue to work
> > > when UIO is not used. Also, this doesn't require any Hyper-V changes.
> > 
> > Hyper-v changes are not the issue here :)
> > 
> > Again, you have to support the situation where a system upgrades to a new
> > kernel and all the same functionality must be there.  How will you do that if
> > you remove drivers from a newer kernel?
> 
> Currently there is a hv_kvp_daemon that interacts with the /dev/device 
> created by the hv_kvp kernel driver, this is the only program interacts with
> this device. This program is acting like a user-space driver, in a sense.
> 
> With the new kernel, if the KVP kernel mode is not present, this kvp daemon
> will not start. All the KVP functionality is handled by the new UIO driver.

And those changes have already been implemented and pushed out
somewhere?

> > > > UIO is for mmaped memory regions, like PCI devices, how is this a
> > > > valid Hyper-V api at all?
> > >
> > > Currently uio_hv_generic is used to mmap VMBUS ring buffers to user-
> > mode.
> > > The primary use case is for DPDK. However, the same mmap model can be
> > > used for all other VMBUS devices, as they all use the same ring buffers
> > model.
> > 
> > Ok, that feels like overkill...
> > 
> > You need to also post the source for these new userspace drivers
> > somewhere for us to review.  I don't see a link to them in the changelog
> > text :(
> 
> Yes, we will share the user space VMBUS drivers. What's the concern of creating
> VMBus driver in user-mode? There are many examples of kernel drivers moving to
> user-mode. For example, DPDK wants a network driver in user-mode,
> SPDK wants a storage driver in user-mode, although they already have corresponding kernel
> drivers.

When moving out of the kernel to userspace, you loose the common and
neutral user/kernel api and are now forced to rely on a userspace
program for access to that device.  no longer can you do something like
normal socket calls, but instead, you have to rely on a library for the
same functionality.

Same for block devices.  Do you really want to give up that common
interface to your block device and rely on a library instead of the
kernel?

If so, great, but don't take away the functionality from all users, as
you will break their systems.

thanks

greg k-h