Hi,
additional info about this:
If the map does not look like e.g.:
host-1 -rw /tmp2 host-1:/tmp2
but instead:
host-1 -rw / host-1:/tmp2
it seems to work perfectly. But we cannot do this without
other problems here. If the map looks like this, what would
be an acceptable workaround:
host-1 -rw / host-1:/tmp2 /tmp2 host-1:/tmp2
the same happens like with the current setup, and it's MUCH
slower. So this is no solution.
Thanks,
Albert
Fluegel Albert wrote:
>
> Hello,
>
> i have a really severe autofs4 problem. Kernel is 2.4.7,
> Redhat-7.0, Hardware is a Fujitsu/Siemens HPC Line dual
> Processor, so kernel is built with SMP turned on. automounter
> Daemon is 4.0.0pre10 . gcc is 2.96:
> Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/2.96/specs
> gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-85)
> RAM is 2 GB, CPU: Pentium III (Coppermine), 256 kb Cache,
> Boot disk is an IDE disk, a SCSI Symbios 53c1010 hostadapter and
> a connected additional disk is installed, but not used, the
> SCSI-adapter driver isn't loaded. IDE controller is
> ServerWorks OSB4.
>
> I'm desperately trying to find the reason for my quite
> reproducible problem (it's not the locking thing in the kernel
> that has been fixed since quite a while)
>
> Assume an map like that:
>
> kaefer -rw /tmp2 kaefer:/tmp2
> gustav -rw /tmp2 gustav:/tmp2
> distel -rw /tmp2 distel:/tmp2
> averna -rw /tmp2 averna:/tmp2
> solp1 -rw /tmp2 solp1:/tmp2
> ...
> host-1 -rw /tmp2 host-1:/tmp2
> host-2 -rw /tmp2 host-2:/tmp2
> ...
>
> i.e. all of those hosts have a /tmp2 directory, that is exported
> properly. Automounter Daemon runs with the following args:
>
> automount -t 1 /mnt/my-subdir yp map-name grpid,hard,nodevs,nosymlink
>
> (or with file instead yp, makes no difference)
>
> Basically we don't want a timeout, which is that short. But it
> helps to reproduce the problem within minutes. Otherwise it
> shows up after some hours, what is not acceptable for users
> with long-running compute jobs. The software accesses the
> /tmp2 directories and home-direcotries (also autofs-Mounted)
> from time to time and once within a few hours the users see
> either a "Permission denied" or "Directory does not exist".
> And this happens with that stress test, too. Assume the following
> script:
>
> #!/bin/sh
>
> # build a list of files to touch within one command
> I=1
> while [ $I -le 32 ] ; do
> FILES="$FILES /mnt/my-subdir/host-$I/tmp2/affile"
> I=`expr $I + 1`
> done
>
> while true ; do
> echo 'trying to touch and ls'
> touch $FILES
> if [ $? -ne 0 ] ; then
> exit 1
> fi
> ls -ld $FILES
>
> date
>
> /bin/rm -f /tmp/bla
> head -4c /dev/random > /tmp/bla
> RANDNUM=`sum /tmp/bla | awk '{print $1}'`
> WAITTIME=`expr '(' $RANDNUM ')' / 2 + 980000`
>
> # random sleep time between 980 ms and ~ 1020 with script
> # interpretatin ... delay - neglible
>
> ls -ld $FILES>/dev/null
> usleep $WAITTIME
> done
>
> What i try to achieve with that random delay is, that
> during expire of the 32 mounts another access to them
> occurs and this in fact happens quite frequently, some-
> times leading to the mentioned error message during
> touch, followed by the exit 1, sometimes not. I wouldn't
> have a problem, if the problem showed up only under this
> stress test, but it's happening also under `normal'
> circumstances. This test runs between few seconds and
> several (say, 20) minutes until failing. It is not
> depending on the kind of NFS-Server. This can be either
> a Linux or Sun machine. No notable difference. I tcpdumped
> the network traffic. No clue. Or better: i can see, that
> no new mount request is leaving the machine directed to
> the server, that supplies one of the directories. Sometimes
> the touch complains about olny one missing directory,
> sometimes about 8 or so, in successive order (e.g.:
> /mnt/my-subdir/host-7/tmp2 through /mnt/my-subdir/host-12/tmp2 )
>
> I suspect, that the autofs4 returns from the revalidate thinking,
> sth has been or is mounted, but has not in reality or has been
> expired in the meantime. I experimented a lot. I modified
> the daemon to serialize the mounts, not just fork and
> report to the kernel, when done. Didn't help. Have a lot
> of strace output about that. Looks all ok. So i suspected,
> it's the kernel autofs4. Found several things, that looked
> weird to me and modified them, but no success. What i tried
> (please don't laugh, maybe some of the stuff is ridiculous):
>
> * protected the queues linked list using a spinlock during
> traversal and modification against parallel accesses
>
> * modified the autofs4_wait function in several more ways
> to have the queues list distinguish between the different
> notification requests. (If thought it might be a problem,
> that a mount request is being added, but finding an expire
> request, thinking 'oh, we have that already, can jump on the
> request for mount', though in fact waiting for expire, but i
> was wrong here, one more time)
>
> * added an additional flag, cause i thought, it might happen
> during execution of the first lines of try_to_fill_dentry,
> where the dcache is not yet locked, that the test for
> AUTOFS_INF_EXPIRING fails and one thread/CPU walks on
> through the function while another CPU works in parallel
> on an ioctl-issued autofs4_expire_multi and before setting
> the flag, the other thread has passed the if. I added another
> flag, that is set in try_to_fill_dentry and checked in
> the autofs4_expire_multi to avoid, that the mountpoint in
> question will be expired, when not expected to expire.
> But this wasn't the problem either.
>
> I have tons of syslog and strace -f output of the daemon, but
> no more clues here.
>
> This problem is quite a show-stopper here, otherwise we must
> set the expire time to infinite, what is not really desirable
> or we must use the userspace amd :-(
>
> Any hint how to narrow the problem is appreciated.
>
> Thanks for reading and with kindest regards,
>
> Albert
>
> --
> Albert Flügel Tel.: +49-89-636-27690
> science+computing AG Fax.: +49-89-636-21950
> bei: Infineon AG D1: +49-171-3698673
> E-Mail: Albert.Fluegel.gp@mchr2.siemens.de
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
-- Albert Flügel Tel.: +49-89-636-27690 science+computing AG Fax.: +49-89-636-21950 bei: Infineon AG D1: +49-171-3698673 E-Mail: Albert.Fluegel.gp@mchr2.siemens.de Technical Project Leader TDSC - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Sun Dec 23 2001 - 21:00:15 EST