Re: [locking/paravirt] static_key_disable_cpuslocked(): static key 'virt_spin_lock_key+0x0/0x20' used before call to jump_label_init()

[Juergen Gross]

On 30/10/17 08:35, Fengguang Wu wrote:
> On Sun, Oct 29, 2017 at 11:51:55PM +0100, Fengguang Wu wrote:
>> Hi Linus,
>>
>> Up to now we see the below boot error/warnings when testing v4.14-rc6.
>>
>> They hit the RC release mainly due to various imperfections in 0day's
>> auto bisection. So I manually list them here and CC the likely easy to
>> debug ones to the corresponding maintainers in the followup emails.
>>
>> boot_successes: 4700
>> boot_failures: 247
> 
> [...]
> 
>> WARNING:at_kernel/jump_label.c:#static_key_disable_cpuslocked: 7

This patch is in the tip tree only, it will be merged in 4.15. So I
don't understand why you are reporting this for 4.14-rc6.

There is a patch by Dou Liyang pending since 28th October addressing
that issue:

[PATCH tip v2] x86/paravirt: Make the virt_spin_lock_key setup after
jump_label_init()


Juergen

> 
> The call trace is
> 
> [    0.000000] Booting paravirtualized kernel on bare hardware
> [    0.000000] clocksource: refined-jiffies: mask: 0xffffffff
> max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
> [    0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:72
> nr_cpu_ids:72 nr_node_ids:2
> [    0.000000] percpu: Embedded 39 pages/cpu @ffff88103f400000 s120088
> r8192 d31464 u262144
> [    0.000000] pcpu-alloc: s120088 r8192 d31464 u262144 alloc=1*2097152
> [    0.000000] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 [0] 08 09 10 11
> 12 13 14 15 [    0.000000] pcpu-alloc: [0] 16 17 36 37 38 39 40 41 [0]
> 42 43 44 45 46 47 48 49 [    0.000000] pcpu-alloc: [0] 50 51 52 53 -- --
> -- -- [1] 18 19 20 21 22 23 24 25 [    0.000000] pcpu-alloc: [1] 26 27
> 28 29 30 31 32 33 [1] 34 35 54 55 56 57 58 59 [    0.000000] pcpu-alloc:
> [1] 60 61 62 63 64 65 66 67 [1] 68 69 70 71 -- -- -- -- [    0.000000]
> static_key_disable_cpuslocked(): static key
> 'virt_spin_lock_key+0x0/0x20' used before call to jump_label_init()
> [    0.000000] ------------[ cut here ]------------
> [    0.000000] WARNING: CPU: 0 PID: 0 at kernel/jump_label.c:161
> static_key_disable_cpuslocked+0x6c/0x80
> [    0.000000] Modules linked in:
> [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
> 4.14.0-rc6-00162-g3d6dabc2 #1
> [    0.000000] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS
> SE5C610.86B.01.01.0008.021120151325 02/11/2015
> [    0.000000] task: ffffffff81e10480 task.stack: ffffffff81e00000
> [    0.000000] RIP: 0010:static_key_disable_cpuslocked+0x6c/0x80
> [    0.000000] RSP: 0000:ffffffff81e03e98 EFLAGS: 00010092 ORIG_RAX:
> 0000000000000000
> [    0.000000] RAX: 000000000000006f RBX: ffffffff81e36fc0 RCX:
> ffffffff81e60cf8
> [    0.000000] RDX: 0000000000000001 RSI: 0000000000000092 RDI:
> 0000000000000047
> [    0.000000] RBP: ffffffff81e03ea8 R08: 6b5f636974617473 R09:
> 000000000000018c
> [    0.000000] R10: 0000000000000000 R11: 000000000000006f R12:
> ffff88207ffcf4c0
> [    0.000000] R13: ffffffff82170920 R14: ffffffff8218b780 R15:
> 0000000000000000
> [    0.000000] FS:  0000000000000000(0000) GS:ffff88103f400000(0000)
> knlGS:0000000000000000
> [    0.000000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    0.000000] CR2: ffff88207f4e0000 CR3: 000000207ee09000 CR4:
> 00000000000606b0
> [    0.000000] Call Trace:
> [    0.000000]  static_key_disable+0x1a/0x30
> [    0.000000]  native_pv_lock_init+0x1b/0x1e
> [    0.000000]  native_smp_prepare_boot_cpu+0x32/0x35
> [    0.000000]  start_kernel+0x14f/0x421
> [    0.000000]  x86_64_start_reservations+0x2a/0x2c
> [    0.000000]  x86_64_start_kernel+0x72/0x75
> [    0.000000]  secondary_startup_64+0xa5/0xb0
> [    0.000000] Code: 85 c0 75 2f 48 c7 c7 20 5e f0 81 e8 df 2a 7b 00 5b
> 41 5c 5d c3 48 89 fa 48 c7 c6 40 93 a3 81 48 c7 c7 00 8b cb 81 e8 95 b5
> f3 ff <0f> ff eb a8 0f ff eb b3 48 89 df e8 14 fc ff ff eb c7 66 90 e8
> [    0.000000] ---[ end trace c12d07f00399ce78 ]---
> [    0.000000] Built 2 zonelists, mobility grouping on.  Total pages:
> 33006159
> [    0.000000] Policy zone: Normal
> 
> which is bisected to
> 
> commit 9043442b43b1fddf202591b84702863286700c1a
> Author:     Juergen Gross <jgross@xxxxxxxx>
> AuthorDate: Wed Sep 6 19:36:24 2017 +0200
> Commit:     Ingo Molnar <mingo@xxxxxxxxxx>
> CommitDate: Tue Oct 10 11:50:12 2017 +0200
> 
>    locking/paravirt: Use new static key for controlling call of
> virt_spin_lock()
> 
>    There are cases where a guest tries to switch spinlocks to bare metal
>    behavior (e.g. by setting "xen_nopvspin" boot parameter). Today this
>    has the downside of falling back to unfair test and set scheme for
>    qspinlocks due to virt_spin_lock() detecting the virtualized
>    environment.
> 
>    Add a static key controlling whether virt_spin_lock() should be
>    called or not. When running on bare metal set the new key to false.
> 
>    Signed-off-by: Juergen Gross <jgross@xxxxxxxx>
>    Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
>    Acked-by: Waiman Long <longman@xxxxxxxxxx>
>    Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
>    Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
>    Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
>    Cc: akataria@xxxxxxxxxx
>    Cc: boris.ostrovsky@xxxxxxxxxx
>    Cc: chrisw@xxxxxxxxxxxx
>    Cc: hpa@xxxxxxxxx
>    Cc: jeremy@xxxxxxxx
>    Cc: rusty@xxxxxxxxxxxxxxx
>    Cc: virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
>    Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx
>    Link: http://lkml.kernel.org/r/20170906173625.18158-2-jgross@xxxxxxxx
>    Signed-off-by: Ingo Molnar <mingo@xxxxxxxxxx>
> ---
> arch/x86/include/asm/qspinlock.h | 11 ++++++++++-
> arch/x86/kernel/paravirt.c       | 14 ++++++++++++--
> arch/x86/kernel/smpboot.c        |  2 ++
> 3 files changed, 24 insertions(+), 3 deletions(-)
> 
> Thanks,
> Fengguang