Re: [PATCH 6/7] x86: mm: set TLB flush tunable to sane value

[Davidlohr Bueso]

On Fri, 2014-03-07 at 09:15 -0800, Dave Hansen wrote:
> On 03/06/2014 05:55 PM, Davidlohr Bueso wrote:
> > On Wed, 2014-03-05 at 16:45 -0800, Dave Hansen wrote:
> >> From: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
> >>
> >> Now that we have some shiny new tracepoints, we can actually
> >> figure out what the heck is going on.
> >>
> >> During a kernel compile, 60% of the flush_tlb_mm_range() calls
> >> are for a single page.  It breaks down like this:
> > 
> > It would be interesting to see similar data for opposite workloads with
> > more random access patterns. That's normally when things start getting
> > fun in the tlb world.
> 
> First of all, thanks for testing.  It's much appreciated!
> 
> Any suggestions for opposite workloads?

I was actually thinking of ebizzy as well.

> I've seen this tunable have really heavy effects on ebizzy.  It fits
> almost entirely within the itlb and if we are doing full flushes, it
> eats the itlb and increases the misses about 10x.  Even putting this
> tunable above 500 pages (which is pretty insane) didn't help it.

Interesting, I didn't expect the misses to be as severe. So I guess what
you say is that this issue is seen even with how we currently have
things.


> Things that thrash the TLB don't really care if someone invalidates
> their TLB since they're thrashing it anyway.

That's a really good point.

> I've had a really hard time finding workloads that _care_ or are
> affected by small changes in this tunable.  That's one of the reasons I
> tried to simplify it: it's just not worth the complexity.

I agree, since we aren't seeing much performance differences anyway I
guess it simply doesn't matter. I can see it perhaps as a factor for
virtualized workloads in the pre-tagged tlb era but not so much
nowadays. In any case I've also asked a colleague to see if he can
produce any interesting results with this patchset on his kvm workloads
but don't expect much surprises.

So all in all I definitely like this cleanup, and things are simplified
significantly without any apparent performance hits. The justification
for the ceiling being 33 seems pretty prudent, and heck, it can be
modified anyway by users. An additional suggestion would be to comment
this magic number, in the code.

Thanks,
Davidlohr

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/