Re: [PATCH 19/22] KVM: x86/mmu: Add infrastructure to allow walking rmaps outside of mmu_lock

[Sean Christopherson]

On Tue, Aug 13, 2024, Lai Jiangshan wrote:
> On Mon, Aug 12, 2024 at 11:22 PM Sean Christopherson <seanjc@xxxxxxxxxx> wrote:
> 
> >
> > Oh yeah, duh, re-read after PAUSE, not before.
> >
> > Definitely holler if you have any alternative ideas for walking rmaps
> > without taking mmu_lock, I guarantee you've spent more time than me
> > thinking about the shadow MMU :-)
> 
> We use the same bit and the same way for the rmap lock.
> 
> We just use bit_spin_lock() and the optimization for empty rmap_head is
> handled out of kvm_rmap_lock().

Hmm, I'm leaning towards keeping the custom locking.  There are a handful of
benefits, none of which are all that meaningful on their own, but do add up.

 - Callers don't need to manually check for an empty rmap_head.
 - Can avoid the redundant preempt_{disable,enable}() entirely in the common case
   of being called while mmu_lock is held.
 - Handles the (likely super rare) edge case where a read-only walker encounters
   an rmap that was just emptied (rmap_head->val goes to zero after the initial
   check to elide the lock).
 - Avoids an atomic when releasing the lock, and any extra instructions entirely
   for writers since they always write the full rmap_head->val when releasing.

> bit_spin_lock() has the most-needed preempt_disable(). I'm not sure if the
> new kvm_rmap_age_gfn_range_lockless() is called in a preempt disabled region.

Oof, it doesn't.  Disabling IRQs crossed my mind, but I completely forgot about
preemption.

Thanks much!