Re: [PATCH, v2] mm, numa: Turn 4K pte NUMA faults into effectivehugepage ones

[Mel Gorman]

On Tue, Nov 20, 2012 at 05:52:39PM +0100, Ingo Molnar wrote:
> 
> * Rik van Riel <riel@xxxxxxxxxx> wrote:
> 
> > Performance measurements will show us how much of an impact it 
> > makes, since I don't think we have never done apples to apples 
> > comparisons with just this thing toggled :)
> 
> I've done a couple of quick measurements to characterise it: as 
> expected this patch simply does not matter much when THP is 
> enabled - and most testers I worked with had THP enabled.
> 
> Testing with THP off hurst most NUMA workloads dearly and tells 
> very little about the real NUMA story of these workloads. If you 
> turn off THP you are living with a constant ~25% regression - 
> just check the THP and no-THP numbers I posted:
> 
>                 [ 32-warehouse SPECjbb test benchmarks ]
> 
>       mainline:                 395 k/sec
>       mainline +THP:            524 k/sec
> 
>       numa/core +patch:         512 k/sec     [ +29.6% ]
>       numa/core +patch +THP:    654 k/sec     [ +24.8% ]
> 
> The group of testers who had THP disabled was thus very low - 
> maybe only Mel alone? The testers I worked with all had THP 
> enabled.
> 
> I'd encourage everyone to report unusual 'tweaks' done before 
> tests are reported - no matter how well intended the purpose of 
> that tweak.

Jeez, it was an oversight. Do I need to send a card or something?

> There's just so many config variations we can test 
> and we obviously check the most logically and most scalably 
> configured system variants first.
> 

I managed to not break the !THP case for the most part in balancenuma
for the cases I looked at. As stated elsewhere not all machines can
support THP that care about HPC -- ppc64 is a major example.  THPs are
not always available, particularly on the node you are trying to migrate
to is fragmented. You can just fail the migration in this case of course
but unless you are willing to compact, this situation can persist for a
long time. You get to keep THP but on a remote node. If we are to ever
choose to split THP to get better placement then we must be able to cope
with the !THP case from the start. Lastly, not all workloads can use THP
if they depend heavily on large files or shared memory.

Focusing on the THP case initially will produce better figures but I worry
it will eventually kick us in the shins and be hard to back out of.

-- 
Mel Gorman
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/