Re: kernel BUG at /data/lemmy/linux.trees.git/fs/nfs/idmap.c:681!

[Myklebust, Trond]

On Thu, 2012-09-27 at 09:59 -0700, Linus Torvalds wrote:
> On Thu, Sep 27, 2012 at 9:16 AM, Myklebust, Trond
> <Trond.Myklebust@xxxxxxxxxx> wrote:
> >
> > I cannot see how that BUG_ON can be triggered in the current code, given
> > that the only place where idmap->idmap_key_cons is set to a non-NULL
> > value is covered by a mutex, and that it is always cleared before we
> > release said mutex.
> 
> Quite frankly, the "I cannot see" thing is *never* an excuse for a BUG_ON().
> 
> We don't do kernel-killing asserts in Linux. Never.
> 
> The only excuse for a BUG_ON() is "I cannot possibly continue, I don't
> even have an error path I can take".
> 
> If it's a fundamentally impossible situation, the BUG_ON() should
> never have been there in the first place!
> 
> And if it's a "I don't see how it could happen", then it should have
> been something like
> 
>     if (WARN_ON_ONCE(condition))
>         goto cleanup;
> 
> rather than a BUG_ON().
> 
> We have too many f*cking BUG_ON's in the kernel, and the fact that one
> triggers and it has taken a month and a half without it even being
> resolved is a problem.
> 
> Get rid of the thing, already, dammit. If you cannot figure out how it
> can happen, then the *last* thing you want to do is then kill the
> machine so that it's impossible to debug it sanely.
> 
> Besides, as far as I can tell, idmap_key_cons locking is suspect
> anyway. Stuff like this:
> 
>                 cons = ACCESS_ONCE(idmap->idmap_key_cons);
>                 idmap->idmap_key_cons = NULL;
> 
> is an almost certain example of "the code is racy, and we did it
> wrong". The above is basically *never* correct.
> 
> If the access is properly locked, then the ACCESS_ONCE() is a bug.
> 
> And if the access *isn't* properly locked, then setting things to NULL
> afterwards is in no way safe.
> 
> IOW, either way, it's broken. And there's at least two of those
> clearly buggy code-sequences involving that field.
> 
> So get rid of the BUG_ON() (possibly replacing it with the
> WARN_ON_ONCE), and please look at those ACCESS_ONCE() sequences and
> fix them. Either they happen under a lock, or they don't. None of this
> crazy racy crap, please.

The ACCESS_ONCE is a bug. It isn't needed, since the whole upcall is
covered by the idmap_mutex. I'll remove it and the BUG_ON() come the
merge window (or I can send a patch sooner if you care).

The downcall is ordered w.r.t. idmap_pipe_destroy_msg() (which clears
idmap->idmap_key_cons only on upcall failure). The downcall is also
protected against collisions with idmap_release_pipe() by means of the
inode->i_mutex. Finally, the call to request_key() (during which the
idmap_mutex is held) will always do an uninterruptible wait for one of
those 3 functions to complete.
IOW: all the functions that manipulate idmap->idmap_key_cons are
theoretically ordered w.r.t. each other. That's what I mean when I say
that I really don't understand why this is happening.

So the first thing to do is to try a _vanilla_ 3.6-rc7, and see if the
problem is reproducible without Joerg's extra patches.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@xxxxxxxxxx
www.netapp.com
N‹§²æìr¸›yúèšØb²X¬¶ÇvØ^–)Þ{.nÇ+‰·¥Š{±‘êçzX§¶›¡Ü}©ž²ÆzÚ&j:+v‰¨¾«‘êçzZ+€Ê+zf£¢·hšˆ§~†­†Ûiÿûàz¹®w¥¢¸?™¨è­Ú&¢)ßf”ù^jÇy§m…á@A«a¶Úÿ0¶ìh®å’i