Re: negative dentry_stat.nr_unused causes aggressive dcache pruning

[Steve Lord]

Andrew Morton wrote:
> Steve Lord <lord@xxxxxxx> wrote:

> > I still do not know exactly how the count gets negative, but I tracked it
> > down to a user space app from emulex called HBAanywhere. The only thing I
> > can see this doing which might be related is attempting to open a lot of
> > non-existant /proc entries:
> >
> > 	/proc/scsi//120
> > 	/proc/scsi//121
> > 	etc...
> >
> > Yes there is a // in there.
> >
> > I ran with a BUG call if we manipulate nr_unused without the dcache lock
> > and it never tripped. All very wierd.
>
>
> Is that 2.4 or 2.6?
>
> I'd be expecting a systematic counting bug.  After all, nr_unused would
> normally be in the thousands and it'd take a lot of races to get that down
> to zero.

 2.4.20-31.9 (aka Yea Olde Kernel), it is a redhat 9.0 update from the
 fedora legacy project.

Something is pushing down hard on the dcache, I can watch the numbers
drop after boot up. They start out a few hundred but within a few
minutes they go negative.

I have a fiber channel driver from out the tree loaded (emulex lpfcdd)
but it does not appear to mess with dcache entries itself. Nothing else
from outside the tree. I have not had a chance to try 2.6 yet, not
even sure this code would run on 2.6.

I don't have the source of this app, so all I can do is strace it and
watch that. Actually I don't even need the app, so I can make the problem
go away, but it seems very odd that a user space program can do this.

If I start up the program and watch /proc/sys/fs/dentry-state every second
or so I see a sequence something like this:
slord@k4> cat /proc/sys/fs/dentry-state
368     22      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
364     18      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
363     16      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
361     14      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
361     14      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
359     13      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
359     13      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
359     13      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
359     13      45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
348     5       45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
332     -16     45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
332     -16     45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
332     -16     45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
332     -16     45      0       0       0
slord@k4> cat /proc/sys/fs/dentry-state
332     -16     45      0       0       0

Its not like I am low on memory though:
slord@k4> free
             total       used       free     shared    buffers     cached
Mem:       1030108     244932     785176          0      15012     147808
-/+ buffers/cache:      82112     947996
Swap:      1574360         76    1574284

All I can see the program doing is attempting to open these non-existent
/proc entries and doing an ioctl FIBMAP on a /dev/ device bound to the fiber
channel driver. Of course, FIBMAP is not FIBMAP in this driver.

It looks like a steaming pile in there, and I think I am just going to
assume the driver is guilty until proven innocent.


Steve


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/