Re: [PATCH] sched/core: Fix kick offline cpu to do nohz idle load balance

From: Wanpeng Li
Date: Mon Oct 10 2016 - 04:53:10 EST


2016-10-10 12:10 GMT+08:00 Wanpeng Li <kernellwp@xxxxxxxxx>:
> From: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>
>
> WARNING: CPU: 0 PID: 3404 at arch/x86/kernel/smp.c:125 native_smp_send_reschedule+0x3f/0x50
> CPU: 0 PID: 3404 Comm: qemu-system-x86 Not tainted 4.8.0+ #21
> Call Trace:
> __warn+0xd1/0xf0
> warn_slowpath_null+0x1d/0x20
> native_smp_send_reschedule+0x3f/0x50
> trigger_load_balance+0x29c/0x4a0
> ? trigger_load_balance+0x72/0x4a0
> scheduler_tick+0x9f/0xd0
> ? tick_sched_do_timer+0x50/0x50
> update_process_times+0x47/0x60
> tick_sched_handle.isra.24+0x25/0x60
> tick_sched_timer+0x3d/0x70
> __hrtimer_run_queues+0xf4/0x510
> hrtimer_interrupt+0xb7/0x1d0
> local_apic_timer_interrupt+0x35/0x60
> smp_apic_timer_interrupt+0x3d/0x50
> apic_timer_interrupt+0x96/0xa0
>
> If there is a need to kick the idle load balancer, an ILB will be selected
> to perform nohz idle load balance, however, if the selected ILB is in the
> process of offline, smp_sched_reschedule() which generates a sched IPI will
> splat as above.
>
> CPU0 CPU1
>
> find_new_ilb()
> set_rq_offline()
> smp_sched_reschedule() Oops
> nohz_balance_exit_idle()
>
> This patch fix it by exiting nohz idle balance before set cpu offline.

CPU 0 CPU1

find_new_ilb()
nohz_balance_exit_idle()
set_rq_offline()
smp_sched_reschedule()

It seems that the patch still can't avoid this race, so any proposal
is a great appreciated. :)

Regards,
Wanpeng Li