RE: [RFC PATCH 03/11] Drivers: hv: vmbus: Replace the per-CPU channel lists with a global array of channels

[Michael Kelley]

From: Andrea Parri <parri.andrea@xxxxxxxxx> Sent: Saturday, March 28, 2020 11:22 AM
> 
> > Correct me if I'm wrong, but currently vmbus_chan_sched() accesses
> > per-cpu list of channels on the same CPU so we don't need a spinlock to
> > guarantee that during an interrupt we'll be able to see the update if it
> > happened before the interrupt (in chronological order). With a global
> > list of relids, who guarantees that an interrupt handler on another CPU
> > will actually see the modified list?
> 
> Thanks for pointing this out!
> 
> The offer/resume path presents implicit full memory barriers, program
> -order after the array store which should guarantee the visibility of
> the store to *all* CPUs before the offer/resume can complete (c.f.,
> 
>   tools/memory-model/Documentation/explanation.txt, Sect. #13
> 
> and assuming that the offer/resume for a channel must complete before
> the corresponding handler, which seems to be the case considered that
> some essential channel fields are initialized only later...)
> 
> IIUC, the spin lock approach you suggested will work and be "simpler";
> an obvious side effect would be, well, a global synchronization point
> in vmbus_chan_sched()...
> 
> Thoughts?
> 

Note that this global array is accessed overwhelmingly with reads.  Once
The system is initialized, channels only rarely come-or-go, so writes will
be rare.  So the array can be cached in all CPUs, and we need to avoid
any global synchronization points.  Leveraging the full semantics of the
memory model (across all architectures) seems like the right approach
to preserve a high level of concurrency.

Michael