Re: dentries: dentry defragmentation

[Nick Piggin]

On Mon, Feb 01, 2010 at 11:22:53AM +0100, Andi Kleen wrote:
> On Mon, Feb 01, 2010 at 09:16:45PM +1100, Nick Piggin wrote:
> > On Mon, Feb 01, 2010 at 11:10:13AM +0100, Andi Kleen wrote:
> > > On Mon, Feb 01, 2010 at 06:08:35PM +1100, Nick Piggin wrote:
> > > > I always preferred to do defrag in the opposite way. Ie. query the
> > > > slab allocator from existing shrinkers rather than opposite way
> > > > around. This lets you reuse more of the locking and refcounting etc.
> > > 
> > > I looked at this for hwpoison soft offline.
> > > 
> > > But it works really badly because the LRU list ordering 
> > > has nothing to do with the actual ordering inside the slab pages.
> > 
> > No, you don't *have* to follow LRU order. The most important thing
> 
> What list would you follow then?

You can follow the slab, as I said in the first mail.

> There's LRU, there's hast (which is as random) and there's slab
> itself. The only one who is guaranteed to match the physical
> layout in memory is slab. That is what this patchkit is trying
> to attempt.
> 
> > is if you followed what I wrote is to get a pin on the objects and
> 
> Which objects? You first need to collect all that belong to a page.
> How else would you do that?

Objects that you're interested in reclaiming, I guess. I don't
understand the question.

 
> > > > whether we are in this slab defrag mode, and if so, whether the slab
> > > > is very sparse. If yes, then reclaim aggressively.
> > > 
> > > The typical result is that you need to get through most of the LRU
> > > list (and prune them all) just to free the page.
> > 
> > Really? If you have a large proportion of slabs which are quite
> > internally fragmented, then I would have thought it would give a
> > significant improvement (aggressive reclaim, that is)
> 
> 
> You wrote the same as me?

Aggressive reclaim: as-in, ignoring referenced bit on the LRU,
*possibly* even trying to actively invalidate the dentry.


> > > > If that doesn't perform well enough and you have to go further and
> > > 
> > > It doesn't.
> > 
> > Can we see your numbers? And the patches you tried?
> 
> What I tried (in some dirty patches you probably don't want to see)
> was to just implement slab shrinking for a single page for soft hwpoison.
> But it didn't work too well because it couldn't free the objects
> still actually in the dcache.
> 
> Then I called the shrinker and tried to pass in the page as a hint
> and drop only objects on that page, but I realized that it's terrible
> inefficient to do it this way.
> 
> Now soft hwpoison doesn't care about a little inefficiency, but I still
> didn't like to be terrible inefficient.
> 
> That is why I asked Christoph to repost his old patchkit that can 
> do the shrink from the slab side (which is the right order here)

Right, but as you can see it is complex to do it this way. And I
think for reclaim driven targetted reclaim, then it needn't be so
inefficient because you aren't restricted to just one page, but
in any page which is heavily fragmented (and by definition there
should be a lot of them in the system).

Hwpoison I don't think adds much weight, frankly. Just panic and
reboot if you get unrecoverable error. We have everything to handle
that so I can't see how it's worth adding much complexity to the
kernel for.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/