Re: [PATCH v8 03/15] futex: Add basic infrastructure for local task local hash.

[Sebastian Andrzej Siewior]

On 2025-02-03 15:27:43 [+0100], Peter Zijlstra wrote:
> On Mon, Feb 03, 2025 at 02:59:23PM +0100, Sebastian Andrzej Siewior wrote:
> > The futex hashmap is system wide and shared by random tasks. Each slot
> > is hashed based on its address and VMA. Due to randomized VMAs (and
> > memory allocations) the same logical lock (pointer) can end up in a
> > different hash bucket on each invocation of the application. This in
> > turn means that different applications may share a hash bucket on the
> > first invocation but not on the second an it is not always clear which
> > applications will be involved. This can result in high latency's to
> > acquire the futex_hash_bucket::lock especially if the lock owner is
> > limited to a CPU and not be effectively PI boosted.
> > 
> > Introduce a task local hash map. The hashmap can be allocated via
> > 	prctl(PR_FUTEX_HASH, PR_FUTEX_HASH_SET_SLOTS, 0)
> > 
> > The `0' argument allocates a default number of 16 slots, a higher number
> > can be specified if desired. The current upper limit is 131072.
> 
> Hmm, I would expect 0 to disable the local thing.
> 
> Now, I realize this is somewhat tricky, since there might be futexes
> inside it. But mapping 0 to some default value seems.. well, odd.

Looking at this from the top of the series:
- The default value (currently 16) supposed to be something sane so the
  user does not have to worry about it. We could increase it if needed.

- Currently each thread will alter the limit if needed based on the
  formula. So even if you set it to 2 or 16, once you have 16 threads
  you will have 64 buckets.

- The number of buckets can only be increased, not shrunk. There is no
  technical requirement for it other than "avoid a race where you use a
  lower number if a lot of threads a fired at once".

- It is not expected to go back to the global hash.

- The upper limit is the same limit as for the global hash.

All the things I mentioned are open for debate:

- We could force the user to specify a power-of-number for the buckets
  (due to current implementation) and not round.

- We could allow to increase and shrink the number of buckets. Allowing
  it to shrink by accident if many threads are fired at once.

- We could say the user knows best and disable the heuristic once a size
  has been requested.

- We could use 0 to disable the local hash and use the global one
  instead if this is a requirement. Doing this before a thread is
  created is preferred. Later might get tricky.

Sebastian