Re: vbus design points: shm and shm-signals

[Gregory Haskins]

Gregory Haskins wrote:
> Anthony Liguori wrote:
>> Fundamentally, how is this different than the virtio->add_buf concept?
> 
> From my POV, they are at different levels.  Calling vbus->shm() is for
> establishing a shared-memory region including routing the memory and
> signal-path contexts.  You do this once at device init time, and then
> run some algorithm on top (such as a virtqueue design).
> 
> virtio->add_buf() OTOH, is a run-time function.  You do this to modify
> the shared-memory region that is already established at init time by
> something like vbus->shm().  You would do this to queue a network
> packet, for instance.
> 
> That said, shm-signal's closest analogy to virtio would be vq->kick(),
> vq->callback(), vq->enable_cb(), and vq->disable_cb().  The difference
> is that the notification mechanism isn't associated with a particular
> type of shared-memory construct (such as a virt-queue), but instead can
> be used with any shared-mem algorithm (at least, if I designed it properly).
> 
> The closest analogy for vbus->shm() to virtio would be
> vdev->config->find_vqs().  Again, the difference is that the algorithm
> (ring, etc) is not dictated by the call.  You then overlay something
> like virtqueue on top.

BTW: Another way to think of this is that virtio->add_buf() is really
"buffer assignment", whereas "vbus->shm()" is "buffer sharing".  The
former is meant to follow an "assign, consume, re-assign, reclaim"
model, where the changing pointer ownership implicitly serializes the
writability of the buffer.  Its used (quite effectively) for things like
passing a network-packet around.

Conversely, the latter case ("buffer sharing") is designed for
concurrent writers.  Its used for things like ring-metadata,
shared-table designs, etc.  Anything that generally is designed for a
longer-term, parallel update model, instead of a consume/reclaim model.

Whether we realize it or not, we generally build buffer-assignment
algorithms on top of buffer-sharing algorithms.  Therefore, while virtio
technically has both of these components, it only exposes the former
(buffer-assignment) as a user-extensible ABI (vq->add_buf).  The latter
(buffer-sharing) is inextricably linked to the underlying virtqueue ABI
(vdev->find_vqs) (or, at least it is today).

This is why I keep emphasizing that they are different layers of the
same stack.  From a device point of view, virtio adds a robust ring
model with buffer-assignment capabilities, support for scatter-gather,
etc.  Vbus underneath it provides a robust buffer-sharing design with
considerations for things like end-to-end prioritization, mitigation of
various virt-like inefficiencies (hypercalls, interrupts, eois, spurious
re-signals), etc.

The idea is you can then join the two together to do something like
build 8-rx virtqueues for your virtio-net to support prio.  If you take
these things into consideration on the backend design as well, you can
actually tie it in end-to-end to gain performance and capabilities not
previously available in KVM (or possibly any virt platform).

HTH,

Kind Regards,
-Greg

Attachment: signature.asc
Description: OpenPGP digital signature