Re: x86_64: Make sparsemem/vmemmap the default memory model

From: Christoph Lameter
Date: Tue Nov 13 2007 - 16:52:48 EST


On Tue, 13 Nov 2007, Jörn Engel wrote:

> On Mon, 12 November 2007 20:41:10 -0800, Christoph Lameter wrote:
> > On Mon, 12 Nov 2007, Ray Lee wrote:
> >
> > > Discontig obviously needs to die. However, FlatMem is consistently
> > > faster, averaging about 2.1% better overall for your numbers above. Is
> > > the page allocator not, erm, a fast path, where that matters?
> > >
> > > Order Flat Sparse % diff
> > > 0 639 641 0.3
> >
> > IMHO Order 0 currently matters most and the difference is negligible
> > there.
>
> Is it? I am a bit concerned about the non-monotonic distribution.
> Difference starts a near-0, grows to 4.4, drops to near-0, grows to 4.9,
> drops to near-0.

The problem also is that the comparison here is between a SMP config for
flatmem vs a NUMA config for sparsemem. There is additional overhead in
the NUMA config.

The effect may also be due to the system being able to place
some pages in the same 2MB section as the memmap with flatmem. However,
that is only feasable immeidately after bootup. In regular operations this
should vanish.

Could you run your own test to verify?

> Is there an explanation for this behaviour? More to the point, could
> repeated runs also return 4% difference for order-0?

I hope I have given some above. The number of the page allocator suggests
that we have far too much fat in the allocation paths. IMHO reasonable
numbers for an order-0 alloc should be ~100 cycles.