RE: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries

[Wangkai (Kevin,C)]

> -----Original Message-----
> From: Waiman Long [mailto:longman@xxxxxxxxxx]
> Sent: Monday, August 21, 2017 9:35 PM
> To: Wangkai (Kevin,C); Alexander Viro; Jonathan Corbet
> Cc: linux-kernel@xxxxxxxxxxxxxxx; linux-doc@xxxxxxxxxxxxxxx;
> linux-fsdevel@xxxxxxxxxxxxxxx; Paul E. McKenney; Andrew Morton; Ingo Molnar;
> Miklos Szeredi; Matthew Wilcox; Larry Woodman; James Bottomley
> Subject: Re: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries
> 
> On 08/20/2017 11:23 PM, Wangkai (Kevin,C) wrote:
> >
> > Yes, I have add some trace info for the dentry state changed, with dentry flag
> and reference count:
> >
> > File create:
> > [   42.636675] dentry [xxxx_1234] 0xffff880230be8180 flag 0x0 ref 1 ev
> dentry alloc
> > File close:
> > [   42.637421] dentry [xxxx_1234] 0xffff880230be8180 flag 0x4800c0 ref 0
> ev dput called
> >
> > Unlink lookup:
> > [  244.658086] dentry [xxxx_1234] 0xffff880230be8180 flag 0x4800c0 ref
> > 1 ev d_lookup Unlink d_delete:
> > [  244.658254] dentry [xxxx_1234] 0xffff880230be8180 flag 0x800c0 ref
> > 1 ev d_lockref ref 1 Unlink dput:
> > [  244.658438] dentry [xxxx_1234] 0xffff880230be8180 flag 0x800c0 ref
> > 0 ev dput called
> >
> > The end, dentry's flag stay at 0x800c0, but this dentry was not freed,
> > keeped by the dcache as unused, After tens of thousands of the
> > dentries slow down the dentry lookup performance, kernel memory usage
> Keep high.
> >
> > Regards,
> > Kevin
> 
> That is expected. The kernel does not get rid of negative dentries until the
> shrinker is called because of memory pressure. Negative dentries do help to
> improve file lookup performance. However, too much of negative dentries
> suppress the amount of free memory available for other use.
> That is why I send out my patch to limit the number of negative dentries
> outstanding.
> 
I think there are two issue:
1. when a file was removed, the dentry should be deleted, this is as a bug, if the
Dentry memory cannot be reclaimed, there is a memory leak.

2. limit the dentries number can improve when there were lots of files operations,
And all the files were valid.

Regards,
Kevin