Re: [block IO crash] Re: 2.6.39-rc5-git2 boot crashs

[Christoph Lameter]

On Wed, 4 May 2011, Pekka Enberg wrote:

> > Well, it's pretty simple:
> >
> > CONFIG_X86_CMPXCHG64=y compiles this_cpu_generic_cmpxchg_double() into:
> >
> >    28f7:       89 f9                   mov    %edi,%ecx
> >    28f9:       8b 3e                   mov    (%esi),%edi
> >    28fb:       89 45 e4                mov    %eax,-0x1c(%ebp)
> >    28fe:       89 c3                   mov    %eax,%ebx
> >    2900:       8b 45 f0                mov    -0x10(%ebp),%eax
> >    2903:       64 0f c7 0f             cmpxchg8b %fs:(%edi)
> >    2907:       0f 94 c0                sete   %al
> >    290a:       84 c0                   test   %al,%al
> >    290c:       88 45 e4                mov    %al,-0x1c(%ebp)
> >    290f:       74 a4                   je     28b5 <kmem_cache_alloc+0x29>
> >
> > while CONFIG_X86_CMPXCHG64=n results in:
> >
> >    28b0:       8b 03                   mov    (%ebx),%eax
> >    28b2:       64 8b 30                mov    %fs:(%eax),%esi
> >    28b5:       39 d6                   cmp    %edx,%esi
> >    28b7:       75 d2                   jne    288b <kmem_cache_alloc+0x26>
> >    28b9:       64 8b 50 04             mov    %fs:0x4(%eax),%edx
> >    28bd:       39 ca                   cmp    %ecx,%edx
> >    28bf:       75 ca                   jne    288b <kmem_cache_alloc+0x26>
> >    28c1:       8b 4b 14                mov    0x14(%ebx),%ecx
> >    28c4:       8b 0c 0e                mov    (%esi,%ecx,1),%ecx
> >    28c7:       64 89 08                mov    %ecx,%fs:(%eax)
> >    28ca:       8b 03                   mov    (%ebx),%eax
> >    28cc:       42                      inc    %edx
> >    28cd:       64 89 50 04             mov    %edx,%fs:0x4(%eax)
> >
> > And that code runs with preemption enabled. So when the task gets
> > preempted _BEFORE_ it has actuallty written back the data, then the
> > race window is wide open.
> >
> > I'm still trying to understand that macro hell which actually
> > generates that code. I always thought that George Anzingers macro maze
> > was horrible, but that's even worse.
>
> I'm adding Christoph and Tejun to CC.

The above should not happen. If a kernel config indicates that there is no
cmpxchg16b/cmpxchg8b available then we need to not compile the path that
uses cmpxchg8b/cmpxchg16b. Guess we need CMPXCHG_DOUBLE_LOCAL or so.