Re: 2.6.26.3 kernel - progressive slowdown over NFS

From: Priyank Patel
Date: Thu Sep 11 2008 - 16:18:14 EST


Resending. Any pointers to other lists or more specific information
which might be required ?

Thanks,
-P

PS : Please cc me on the response.

On Tue, Sep 9, 2008 at 5:47 PM, Priyank Patel <pkpatel.lists@xxxxxxxxx> wrote:
> Hi,
>
>
> We have a simple python program which keeps running a C loop to lstat
> NFS mounted directories. We are seeing some weird behavior w.r.t. the
> run-time of this program on 2.6.26.3 kernel vs 2.6.24 kernel.
>
> The run-time of the following code increases over time on the 2.6.26.3
> kernel, whereas remains flat (as expected) on the 2.6.24 kernel.
> [See attached graphs - B1.jpg and B2.jpg] Once the 2.6.26.3 machine
> gets into this state, we need to restart the box to get back to
> reasonable run-times. Is this a known issue ?
>
> Setup :
>
> Machine A (2.6.26) : exports NFS directory - /a/baz contains 10,000
> directories bar0 ... bar9999
> /a/baz *(rw,sync,no_root_squash,no_all_squash,subtree_check)
>
> Machine B1 (2.6.26.3) : mounts NFS dir RO from A - graph B1.jpg
> 10.x.x.x:/a/baz on /baz type nfs
> (ro,vers=3,rsize=4096,wsize=4096,namlen=255,hard,nointr,nolock,proto=udp,timeo=11,retrans=2,sec=sys,mountproto=udp,addr=10.x.x.x)
>
> Machine B2 (2.6.24) : mounts NFS dir RO from A - graph B2.jpg
> 10.x.x.x:/a/baz on /baz type nfs
> (ro,vers=3,rsize=4096,wsize=4096,hard,nointr,nolock,proto=udp,timeo=11,retrans=2,sec=sys,addr=10.x.x.x)
>
> Repro :
>
> B1 and B2, run the following python program :
> {{{
> #!/usr/bin/env python
>
> import os
> import sys
> import time
>
> while True:
> t1 = time.time()
> rv = os.system("/a.out 10000 >& /dev/null") # lstat the 10000
> directories mounted via NFS
> t2 = time.time()
> print >> sys.stderr, "%.3f" % (t2 - t1), rv
> }}}
>
> where a.out is the following C code :
>
> {{{
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <unistd.h>
>
> int main(int argc, char **argv)
> {
> int i, n = atoi(argv[1]);
> char filename[64];
> struct stat statbuf;
>
> for (i = 0; i < n; i++) {
> sprintf(filename, "/baz/bar%d", i);
> lstat(filename, &statbuf);
> }
>
> printf("done\n");
> return 0;
> }
> }}}
>
> Attachments :
>
> B1.jpg and B2.jpg - runtimes on the 2 client machines -
> X-axis : iteration number
> Y-axis : run-time
>
> Config files : - for the machines B1, B2 and A
>
> b1.2.6.26.3.config
> b2.2.6.24.config
> a.2.6.26.config
>
>
> - P
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/