Re: [RFC][PATCH] make global bitlock waitqueues per-node

[Nicholas Piggin]

On Mon, 19 Dec 2016 16:20:05 -0800
Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx> wrote:

> On 12/19/2016 03:07 PM, Linus Torvalds wrote:
> >     +wait_queue_head_t *bit_waitqueue(void *word, int bit)
> >     +{
> >     +       const int __maybe_unused nid = page_to_nid(virt_to_page(word));
> >     +
> >     +       return __bit_waitqueue(word, bit, nid);
> > 
> > No can do. Part of the problem with the old coffee was that it did that
> > virt_to_page() crud. That doesn't work with the virtually mapped stack.   
> 
> Ahhh, got it.
> 
> So, what did you have in mind?  Just redirect bit_waitqueue() to the
> "first_online_node" waitqueues?
> 
> wait_queue_head_t *bit_waitqueue(void *word, int bit)
> {
>         return __bit_waitqueue(word, bit, first_online_node);
> }
> 
> We could do some fancy stuff like only do virt_to_page() for things in
> the linear map, but I'm not sure we'll see much of a gain for it.  None
> of the other waitqueue users look as pathological as the 'struct page'
> ones.  Maybe:
> 
> wait_queue_head_t *bit_waitqueue(void *word, int bit)
> {
> 	int nid
> 	if (word >= VMALLOC_START) /* all addrs not in linear map */
> 		nid = first_online_node;
> 	else
> 		nid = page_to_nid(virt_to_page(word));
>         return __bit_waitqueue(word, bit, nid);
> }

I think he meant just make the page_waitqueue do the per-node thing
and leave bit_waitqueue as the global bit.

It would be cool if CPUs had an instruction that translates an address
though. You could avoid all that lookup and just do it with the TLB :)

Thanks,
Nick