Re: [PATCH v1 11/17] lib: add light-weight queuing mechanism.

[NeilBrown]

On Tue, 12 Sep 2023, Andrew Morton wrote:
> On Mon, 11 Sep 2023 20:30:40 +0000 Chuck Lever III <chuck.lever@xxxxxxxxxx> wrote:
> 
> > 
> > 
> > > On Sep 11, 2023, at 2:13 PM, Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:
> > > 
> > > On Mon, 11 Sep 2023 10:39:43 -0400 Chuck Lever <cel@xxxxxxxxxx> wrote:
> > > 
> > >> lwq is a FIFO single-linked queue that only requires a spinlock
> > >> for dequeueing, which happens in process context.  Enqueueing is atomic
> > >> with no spinlock and can happen in any context.
> > > 
> > > What is the advantage of this over using one of the library
> > > facilities which we already have?
> > 
> > I'll let the patch author respond to that question, but let me pose
> > one of my own: What pre-existing facilities are you thinking of, so
> > that I may have a look?
> 
> Well, I assume that plain old list_heads could be recruited for this
> requirement.  And I hope that a FIFO could be implemented using kfifo ;)
> 

Plain old list_heads (which the code currently uses) require a spinlock
to be taken to insert something into the queue.  As this is usually in
bh context, it needs to be a spin_lock_bh().  My understanding is that
the real-time developers don't much like us disabling bh.  It isn't an
enormous win switching from a list_head list to a llist_node list, but
there are small gains such as object size reduction and less locking.  I
particularly wanted an easy-to-use library facility that could be
plugged in to two different uses cases in the sunrpc code and there
didn't seem to be one.  I could have written one using list_head, but
llist seemed a better fix.  I think the code in sunrpc that uses this
lwq looks a lot neater after the conversion.

I wasn't aware of kfifo.  Having now looked at it I don't think it would
be suitable.  It is designed to provide a fixed-size buffer for data
with cycling "head" and "tail" pointers - typically for data coming from
or to a device.  It doesn't provide any locking support so we would need
locking both to enqueue and to dequeue.  Thus it would be no better than
the list_head approach, and the different interface style would make it
harder to use (an "impedance mismatch"?).  A simple summary might be
that kfifo provides a buffer, not a queue.

Thanks,
NeilBrown