Re: [llvmlinux] percpu | bitmap issue? (Cannot boot on bare metal due to a kernel NULL pointer dereference)

From: Sedat Dilek
Date: Tue Sep 08 2015 - 23:14:40 EST

On Wed, Sep 9, 2015 at 5:04 AM, Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> On Wed, Sep 9, 2015 at 4:51 AM, Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
>> On Wed, Sep 9, 2015 at 4:29 AM, Baoquan He <bhe@xxxxxxxxxx> wrote:
>>>
>>> Seems it happened when call num_possible_cpus() which is a inline
>>> function to call __bitmap_weight(). So did you check
>>> lib/bitmap.c:__bitmap_weight() by objdump?
>>>
>>> setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
>>>
>>
>> [ CC->TO Denys ]
>>
>> Hi Boaquan,
>>
>> Thanks for your reply.
>>
>> I tried a lot in the meantime :-).
>>
>> For example I applied a patch from Denys Vlasenko <dvlasenk@xxxxxxxxxx>...
>>
>> commit 1a1d48a4a8fde49aedc045d894efe67173d59fe0
>> "linux/bitmap: Force inlining of bitmap weight functions"
>>
>> ...on top of Linux v4.2.
>>
>> This resulted in the same call-trace in QEMU.
>>
>> I hacked around to only re-build mm/percpu.c with GCC (rest with
>> CLANG) with some guidance from Linus (compiler warapper-script) etc.
>>
>> What do you need for outputs?
>> objdump of lib/bitmap.o?
>> From both builds - with GCC and CLANG?
>>
>> Please give me clear instructions, Thanks!
>>
>> Regards,
>> - Sedat -
>>
>> [1] http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=1a1d48a4a8fde49aedc045d894efe67173d59fe0
>
> Below snippets against Linux v4.2 and Linux v4.2 plus llvmlinux-fixes
> (see file-attachment)...
> ( The patch "linux/bitmap: Force inlining of bitmap weight functions"
> from Denys was not applied. )
>
> [ lib-bitmap_o-objdump-drw_CLANG-3-7.txt ]
>
> 0000000000000670 <__bitmap_weight>:
> 670: 49 89 f8 mov %rdi,%r8
> 673: 31 d2 xor %edx,%edx
> 675: 41 89 f1 mov %esi,%r9d
> 678: 41 c1 e9 06 shr $0x6,%r9d
> 67c: b8 00 00 00 00 mov $0x0,%eax
> 681: 74 26 je 6a9 <__bitmap_weight+0x39>
> 683: 31 d2 xor %edx,%edx
> 685: 45 89 ca mov %r9d,%r10d
> 688: 4c 89 c1 mov %r8,%rcx
> 68b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
> 690: 48 8b 39 mov (%rcx),%rdi
> 693: e8 00 00 00 00 callq 698
> <__bitmap_weight+0x28> 694: R_X86_64_PC32
> __sw_hweight64-0x4
> 698: 89 d2 mov %edx,%edx
> 69a: 48 01 c2 add %rax,%rdx
> 69d: 48 83 c1 08 add $0x8,%rcx
> 6a1: 41 ff ca dec %r10d
> 6a4: 75 ea jne 690 <__bitmap_weight+0x20>
> 6a6: 44 89 c8 mov %r9d,%eax
> 6a9: 40 f6 c6 3f test $0x3f,%sil
> 6ad: 74 1c je 6cb <__bitmap_weight+0x5b>
> 6af: f7 de neg %esi
> 6b1: 48 c7 c7 ff ff ff ff mov $0xffffffffffffffff,%rdi
> 6b8: 40 88 f1 mov %sil,%cl
> 6bb: 48 d3 ef shr %cl,%rdi
> 6be: 49 23 3c c0 and (%r8,%rax,8),%rdi
> 6c2: e8 00 00 00 00 callq 6c7
> <__bitmap_weight+0x57> 6c3: R_X86_64_PC32
> __sw_hweight64-0x4
> 6c7: 01 d0 add %edx,%eax
> 6c9: 89 c2 mov %eax,%edx
> 6cb: 89 d0 mov %edx,%eax
> 6cd: c3 retq
> 6ce: 66 90 xchg %ax,%ax
>
> [ lib-bitmap_o-objdump-drw_GCC-4-9.txt ]
>
> 0000000000000d60 <__bitmap_weight>:
> d60: 89 f1 mov %esi,%ecx
> d62: 55 push %rbp
> d63: 49 89 fa mov %rdi,%r10
> d66: c1 e9 06 shr $0x6,%ecx
> d69: 85 c9 test %ecx,%ecx
> d6b: 48 89 e5 mov %rsp,%rbp
> d6e: 74 52 je dc2 <__bitmap_weight+0x62>
> d70: 8d 41 ff lea -0x1(%rcx),%eax
> d73: 48 89 fa mov %rdi,%rdx
> d76: 45 31 c0 xor %r8d,%r8d
> d79: 4c 8d 0c c5 08 00 00 00 lea 0x8(,%rax,8),%r9
> d81: 49 01 f9 add %rdi,%r9
> d84: 48 8b 3a mov (%rdx),%rdi
> d87: 48 83 c2 08 add $0x8,%rdx
> d8b: e8 00 00 00 00 callq d90 <__bitmap_weight+0x30>
> d8c: R_X86_64_PC32 __sw_hweight64-0x4
> d90: 41 01 c0 add %eax,%r8d
> d93: 4c 39 ca cmp %r9,%rdx
> d96: 75 ec jne d84 <__bitmap_weight+0x24>
> d98: 40 f6 c6 3f test $0x3f,%sil
> d9c: 44 89 c0 mov %r8d,%eax
> d9f: 74 1f je dc0 <__bitmap_weight+0x60>
> da1: f7 de neg %esi
> da3: 89 ca mov %ecx,%edx
> da5: 48 c7 c0 ff ff ff ff mov $0xffffffffffffffff,%rax
> dac: 89 f1 mov %esi,%ecx
> dae: 48 d3 e8 shr %cl,%rax
> db1: 49 23 04 d2 and (%r10,%rdx,8),%rax
> db5: 48 89 c7 mov %rax,%rdi
> db8: e8 00 00 00 00 callq dbd <__bitmap_weight+0x5d>
> db9: R_X86_64_PC32 __sw_hweight64-0x4
> dbd: 44 01 c0 add %r8d,%eax
> dc0: 5d pop %rbp
> dc1: c3 retq
> dc2: 85 f6 test %esi,%esi
> dc4: 74 19 je ddf <__bitmap_weight+0x7f>
> dc6: f7 de neg %esi
> dc8: 48 c7 c0 ff ff ff ff mov $0xffffffffffffffff,%rax
> dcf: 45 31 c0 xor %r8d,%r8d
> dd2: 89 f1 mov %esi,%ecx
> dd4: 48 d3 e8 shr %cl,%rax
> dd7: 49 23 02 and (%r10),%rax
> dda: 48 89 c7 mov %rax,%rdi
> ddd: eb d9 jmp db8 <__bitmap_weight+0x58>
> ddf: 31 c0 xor %eax,%eax
> de1: 5d pop %rbp
> de2: c3 retq
> de3: 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 data32 data32 data32
> nopw %cs:0x0(%rax,%rax,1)
>

Looking at the GCC bug-report [1] mentionned in [2] I have set here...

CONFIG_OPTIMIZE_INLINING=y

- Sedat -

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66122
[2] http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=1a1d48a4a8fde49aedc045d894efe67173d59fe0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/