Re: [PATCH] Deadlock during heavy write activity to userspace NFSserver on local NFS mount

[Nick Piggin]

Avi Kivity wrote:
> Pavel Machek wrote:
>
> > I'd hope that kswapd was carefully to make sure that it always has
> > enough pages...
> >
> > ...it is harder to do the same auditing with userland program.
> >
> >  
> Very true. But is a kernel thread like kswapd depends on a userspace 
> program, then that program better be well behaved.
>
> > > A more complete solution would be to assign memory reserve levels 
> > > below which a process starts allocating synchronously. For example, 
> > > normal processes must have >20MB to make forward progress, kswapd 
> > > wants >15MB and the NFS server needs >10MB. Some way would be needed 
> > > to express the dependencies.
> > >   
> >
> >
> > Yes, something like that would be neccessary. I believe it would be
> > slightly more complicated, like
> >
> > "NFS server needs > 10MB *and working kswapd*", so you'd need 25MB in
> > fact... and this info should be stored in some readable form so that
> > it can be checked.
> >
> >  
> If the NFS server needed kswapd, we'd deadlock pretty soon, as kswapd 
> *really* needs the NFS server. In our case, all block I/O is done using 
> unbuffered I/O, and all memory is preallocated, so we don't need kswapd 
> at all, just that small bit of memory that syscalls consume.
>
> If the NFS server really needs kswapd, then there'd better be two of 
> them. Regular processes would depend on one kswapd, which depends on the 
> NFS server, which depends on the second kswapd, which depends on the 
> hardware alone. It should be fun trying to describe that topology to the 
> kernel through some API.
>
> Our filesystem actually does something like that internally, except the 
> dependency chain length is seven, not two.

There is some need arising for a call to set the PF_MEMALLOC flag for
userspace tasks, so you could probably get a patch accepted. Don't
call it KSWAPD_HELPER though, maybe MEMFREE or RECLAIM or RECLAIM_HELPER.

But why is your NFS server needed to reclaim memory? Do you have the
filesystem mounted locally?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/