Re: Severe VFS Performance Issues 2.6 with > 95000 directory entries

[Jeff V. Merkey]

Douglas McNaught wrote:

> jmerkey <jmerkey@xxxxxxxxxxxxxxxxxx> writes:
>
>  
>
> > Nikita Danilov wrote:
> >
> >    
> >
> > > Jeff V. Merkey writes:
> > >      
> > >
> > > > > The subject line speaks for itself.   This is using standard VFS
> > > > >          
> > > readdir > and lookup calls through the VFSwith ftp.  Very poor. 
> > >
> > > Reiser4 works fine with 100M entries in a directory, so VFS is not a
> > > bottleneck here.
> > >
> > >
> > >      
> > how about with ftp running on top? Try running FTP in directory with
> > 100M entries. See how long it takes to return the data to
> > the remote client for a dir listing.
> >    
>
> What filesystem are you using?  If it's ext3 without dirindex turned
> on, that would definitely explain it.
>
>  
I just noticed the I_NEW flag for iget which prevents multiple calls to 
refresh the inode. There's another code section where I update the 
filesize field
after I call iget from lookup. This does not explain it either since I 
use math here to hash and post into the inode. I am still convinced that 
either in userspace
or in the kernel VFS, there's still a case where readdit goes linear and 
starts to exhibit (O)(Log 2(N)) behavior as the directory gets large 
(above 50,000 entries).

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/