Re: [PATCHSET x86/core/percpu] implement dynamic percpu allocator

[Tejun Heo]

Hello, Ingo.

Ingo Molnar wrote:
> * Tejun Heo <tj@xxxxxxxxxx> wrote:
> 
>> Tejun Heo wrote:
>>>   One trick we can do is to reserve the initial chunk in non-vmalloc
>>>   area so that at least the static cpu ones and whatever gets
>>>   allocated in the first chunk is served by regular large page
>>>   mappings.  Given that those are most frequent visited ones, this
>>>   could be a nice compromise - no noticeable penalty for usual cases
>>>   yet allowing scalability for unusual cases.  If this is something
>>>   which can be agreed on, I'll pursue this.
>> I've given more thought to this and it actually will solve 
>> most of issues for non-NUMA but it can't be done for NUMA.  
>> Any better ideas?
> 
> It could be allocated via NUMA-aware bootmem allocations.

Hmmm... not really.  Here's what I was planning to do on non-NUMA.

  Allocate the first chunk using alloc_bootmem().  After setting up
  each unit, give back extra space sans the initialized static area
  and some amount of free space which should be enough for common
  cases by calling free_bootmem().  Mark the returned space as used in
  the chunk map.

This will allow sane chunk size and scalability without adding TLB
pressure, so it's actually pretty sweet.  Unfortunately, this doesn't
really work for NUMA because we don't have control over how NUMA
addresses are laid out so we can't allocate contiguous NUMA-correct
chunk without remapping.  And if we remap, we can't give back what's
left to the allocator.  Giving back the original address doubles TLB
usage and giving back the remapped address breaks __pa/__va.  :-(

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/