Re: stat very inefficient

[Peter Chubb]

>>>>> "viro" == viro  <viro@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> writes:

>> On Wed, 28 Jul 2004 17:08:37 -0700 Chris Wedgwood <cw@xxxxxxxx>
>> wrote:
>> 
>> > Just How bad is it for you?  I just tested stat on my crapbox and
>> for > a short path 1M stats takes 0.5s and for a longer path (30
>> bytes or > so) 2.8s.
>> 
>> Run "time find . -type f" on the kernel tree, both before and after
>> removing the third unnecessary copy.

viro> ... with hot cache, otherwise IO time will dominate.  I don't
viro> disagree with you, but in all realistic cases I can think of
viro> it's going to be noise (e.g. this find over kernel tree is
viro> almost certainly followed by xargs grep, etc.).

With hot cache the system time is really small.

On a 2GHz Pentium 4, Compare
	find .-type f -mtime -2000 >/dev/null
with
	find . -type f -mtime -2000
in a freshly checked out 2.8 kernel tree.

(the -mtime test is to force a stat, otherwise, as Ulrich says, almost
no stat system calls will take place)

	to xterm     	>/dev/null	| xargs grep foo
sys	0.34		0.103		0.35
user    0.29		0.08		0.104
real    18.551		0.204		220.25

Using strace reveals that around 60% of the system time in the
redirected to /dev/null case is lstat64 --- 41465 calls, 1.5usec per
call.  Where it's in a pipe that uses the files, the time is swamped
by the time to process the files, and the time spent in write() ---
lstat64 drops to around 16% of the time in find.

The nice thing about the current three-copy implementation is that
it's simple and obviously correct.  Personally, I don't think that the
increased complexity of arhcitecture-specific callbacks, etc., is
worth the small performance gain.


--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/