Re: [RFC PATCH] vfs: limit directory child dentry retention
From: Ian Kent
Date: Wed Apr 08 2026 - 20:59:48 EST
On 8/4/26 18:17, Jan Kara wrote:
On Tue 07-04-26 20:41:08, Ian Kent wrote:
On 7/4/26 18:35, Christian Brauner wrote:
On Wed, Apr 01, 2026 at 10:10:55AM +0800, Ian Kent wrote:Yes, I did take that on board.
On 31/3/26 17:39, Christian Brauner wrote:I meant when when you start limiting the number of negative dentries and
On Tue, Mar 31, 2026 at 09:29:09AM +0800, Ian Kent wrote:I'm pretty sure I saw that earlier on but had forgotten about it when I
If there's a very large number of children present in a directory dentryFwiw, there's also e6957c99dca5 ("vfs: Add a sysctl for automated deletion of dentry")
then the benifit from retaining stale child dentries for re-use can
become ineffective. Even hashed lookup can become ineffective as hash
chains grow, time taken to umount a file system can increase a lot, as
well as child dentry traversals resulting in lock held too long log
messages.
reviewed the bug this time around. It is essentially 681ce8623567 ("vfs:
Delete the associated dentry when deleting a file") with opt-in of course.
This patch introduces the concept conditionally, where the associatedI meant to update Documentation/admin-guide/sysctl/fs.rst to also say that
dentry is deleted only when the user explicitly opts for it during file
removal. A new sysctl fs.automated_deletion_of_dentry is added for this
purpose. Its default value is set to 0.
setting dir-stale-max to 0 disables it. I also get the impression you might
feel better about this if the default was 0 as well.
The thing that I don't much like with the d_delete() approach is that it
fails to cater for files that have been closed and are otherwise unused
who's dentries make there way to the LRU eventually resulting in the bad
behaviour being discussed.
I have no massive objections to your approach. It feels a bit hacky tbhMy aim was to improve performance so I'm a bit puzzled by the comment.
as it seems to degrade performance for new workloads in favor old
workloads. The LRU should sort this out though.
The problem is the sheer number of dentry objects and the consequences
of that. Hash table chains growing will affect performance, umounting
the mount will take ages, and there are cases of child dentry traversals
in the VFS. Once you get a large number of stale dentries that necessarily
need to stay linked into the structures to get the benefit of caching your
exposed to this problem.
The LRU mechanism is so far unable to cope with this.
start to not accumulate more that some workload was relying on this. It
was just a theoretical musing. That initial "delete on unlink" thing led
to regressions for a bunch of workloads when we did it unconditionally
initially.
It's trivial to initially disable it with a doc update describng this.
But this only works (gets accepted) if people think it's worth while and
I'm not sure my proposed case is convincing enough.
Thanks for the input Jan.
I think we definitely need to find a way to manage negative dentries (and
possibly all dentries) better. It is coming up several times each year that
people have issues with negative dentries accumulating too much and they
hit issues in various places.
Indeed, that's precisely why I've been thinking about it.
I don't particularly like your solution because it effectively stops
caching new dentries for the parent and old ones stay while, as Christian
suggested, we'd prefer to prune older dentries first. Also usually the
problem is with negative dentries while you approach treats negative and
positive dentries in the same way which could be a problem for some cases.
But with this second objection I'm less sure so take that mostly as a
preference :).
Maybe I'm not paying attention but I didn't "get" the new vs. old notion
from previous discussion even though I was thinking the same thing. So good,
you've fixed that, ;)
The second thing you mention is a bit more subtle so I'll need to look again
at the dput() handling for the last reference of a dentry.
The problem is how to efficiently prune older children dentries without
slowing down fast paths or growing struct dentry...
The notion here is the best way to do that is to work out some way to
identify dentries that should be discarded rather than kept (on the
LRU) at final dput.
Very large directories have obvious performance overheads. Such as once
the hash bucket queue lengths grow past a certain ratio of buckets to
queue length (can't remember what ratio I saw in user space autofs when
I had this same problem) d_lookup*() will suffer and the benefit of
retaining a dentry will go away regardless of dentry age. Nevertheless
oldest first is an excellent approach, I'll continue thinking about how
to do that.
Thanks, Ian