Re: [QUESTION] about the maple tree and current status of mmap_lock scalability

[Matthew Wilcox]

On Wed, Dec 28, 2022 at 09:48:51PM +0900, Hyeonggon Yoo wrote:
> Hello mm folks,
> 
> I have a few questions about the current status of mmap_lock scalability.
> 
> =============================================================
> What is currently causing the kernel to use mmap_lock to protect the maple tree?
> =============================================================
> 
> I understand that the long-term goal is to remove the need for mmap_lock in readers
> while traversing the maple tree, using techniques such as RCU or SPF.
> What is the biggest obstacle preventing this from being achieved at this time?

The long term goal is even larger than this.  Ideally, the VMA tree
would be protected by a spinlock rather than a mutex.  That turned out
to be too large a change for the moment (and isn't all that important
compared to enabling RCU readers)

> ==================================================
> How does the maple tree provide RCU-safe manipulation of VMAs?
> ==================================================
> 
> Is it similar to the approach suggested in the RCUVM paper (replacing the original
> root node with a new root node that shares most of its nodes and deferring
> the freeing of stale nodes using RCU)?
> 
> I'm having difficulty understanding the design of the maple tree in this regard.
> 
> [RCUVM paper] https://pdos.csail.mit.edu/papers/rcuvm:asplos12.pdf

While I've read the RCUVM paper, I wouldn't say it was particularly an
inspiration.  The Maple Tree is independent of the VM; it's a general
purpose B-tree.  As with any B-tree, when modifying a node, we don't
touch nodes that we don't need to touch.  As with any RCU data structure,
we defer freeing things while RCU readers might still have a reference
to them.

We don't necessarily go all the way to the root node when modifying a
leaf node.  For example, if we have this structure:

Root: Node A, 4000, Node B
Node A: p1, 50, p2, 100, p3, 150, p4, 200, NULL, 250, p6, 1000, p7
Node B: p8, 4050, p9, 4100, p10, 4150, p11, 4200, NULL, 4250, p13

and we replace p4 with a NULL over the whole range from 150-199,
we construct a new Node A2 that contains:

Node A2: p1, 50, p2, 100, p3, 150, NULL, 250, p6, 1000, p7

and we simply write A2 over the entry in Root.  Then we mark Node A as
dead and RCU-free Node A.  There's no need to replace Root as stores
to a pointer are atomic.  If we need to rebalance between Node A and
Node B, we will need to create a new Root (as well as both A and B),
mark all of them as dead and RCU-free them.