Re: [RFC PATCH v2] ptr_ring: linked list fallback

[Jason Wang]

On 2018å02æ28æ 22:01, Michael S. Tsirkin wrote:
> On Wed, Feb 28, 2018 at 02:28:21PM +0800, Jason Wang wrote:
> > On 2018å02æ28æ 12:09, Michael S. Tsirkin wrote:
> > > > > Or we can add plist to a union:
> > > > >
> > > > >
> > > > > struct sk_buff {
> > > > >            union {
> > > > >                    struct {
> > > > >                            /* These two members must be first. */
> > > > >                            struct sk_buff          *next;
> > > > >                            struct sk_buff          *prev;
> > > > >                            union {
> > > > >                                    struct net_device       *dev;
> > > > >                                    /* Some protocols might use this space to store information,
> > > > >                                     * while device pointer would be NULL.
> > > > >                                     * UDP receive path is one user.
> > > > >                                     */
> > > > >                                    unsigned long           dev_scratch;
> > > > >                            };
> > > > >                    };
> > > > >                    struct rb_node  rbnode; /* used in netem & tcp stack */
> > > > > +		struct plist plist; /* For use with ptr_ring */
> > > > >            };
> > > > This look ok.
> > > >
> > > > > > For XDP, we need to embed plist in struct xdp_buff too,
> > > > > Right - that's pretty straightforward, isn't it?
> > > > Yes, it's not clear to me this is really needed for XDP consider the lock
> > > > contention it brings.
> > > >
> > > > Thanks
> > > The contention is only when the ring overflows into the list though.
> > Right, but there's usually a mismatch of speed between producer and
> > consumer. In case of a fast producer, we may get this contention very
> > frequently.
> >
> > Thanks
> This is not true in my experiments.  In my experiments, ring size of 4k
> bytes is enough to see packet drops in single %s of cases.
>
> To you have workloads where rings are full most of the time?

E.g using xdp_redirect to redirect packets from ixgbe to tap. In my 
test, ixgeb can produce ~8Mpps. But vhost can only consume ~3.5Mpps.

> One other nice side effect of this patch is that instead of dropping
> packets quickly it slows down producer to match consumer speeds.

In some case, producer may not want to be slowed down, e.g in devmap 
which can redirect packets into several different interfaces.

> IOW, it can go either way in theory, we will need to test and see the effect.

Yes.

Thanks