Re: Kernel SCM saga..

[Linus Torvalds]

On Fri, 8 Apr 2005, Chris Wedgwood wrote:
> 
> > It doesn't matter so much for the cached case, but it _does_ matter
> > for the uncached one.
> 
> Doing the minimal stat cold-cache here is about 6s for local disk.
> I'm somewhat surprised it's that bad actually.

One of the reasons I do inode numbers in the "index" file (apart from 
checking that the inode hasn't changed) is in fact that "stat()" is damn 
slow if it causes seeks. Since your stat loop is entirely 

You can optimize your stat() patterns on traditional unix-like filesystems
by just sorting the stats by inode number (since the inode number is
historically a special index into the inode table - even when filesystems
distribute the inodes over several tables, sorting will generally do the
right thing from a seek perspective). It's a disgusting hack, but it
literally gets you orders-of-magnitude performance improvments in many
real-life cases.

It does have some downsides:
 - it buys you nothing when it's cached (and obviously you have the 
   sorting overhead, although that's pretty cheap)
 - on other filesystems it can make things slower.

But if the cold-cache case actually is a concern, I do have the solution 
for it. Just a simple "prime-cache" program that does a qsort on the index 
file entries and does the stat() on them all will bring the numbers down. 
Those 6 seconds you see are the disk head seeking around like mad.

			Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/