RE: [PATCH] Kick CPUS that might be sleeping in cpus_idle_wait

[Pallipadi, Venkatesh]

>-----Original Message-----
>From: Steven Rostedt [mailto:rostedt@xxxxxxxxxxx] 
>Sent: Wednesday, January 09, 2008 12:42 PM
>To: LKML
>Cc: Linus Torvalds; Andrew Morton; Ingo Molnar; Thomas 
>Gleixner; Brown, Len; Pallipadi, Venkatesh; Adam Belay; Peter 
>Zijlstra; Andi Kleen
>Subject: [PATCH] Kick CPUS that might be sleeping in cpus_idle_wait
>
>This patch is different than the first patch I sent out.
>This one just sends an IPI to all CPUS that don't check in after 1 sec.
>
>
>Sometimes cpu_idle_wait gets stuck because it might miss CPUS that are
>already in idle, have no tasks waiting to run and have no interrupts
>going to them.  This is common on bootup when switching cpu idle
>governors.
>
>This patch gives those CPUS that don't check in an IPI kick.
>

I think your RFC patch is the right solution here. As I see it, there is
no race with your RFC patch.
As long as you call a dummy smp_call_function on all CPUs, we should be
OK. We can get rid of cpu_idle_state
and the current wait forever logic altogether with dummy
smp_call_function. And so there wont be any wait forever scenario.

The whole point of cpu_idle_wait() is to make all CPUs come out of idle
loop atleast once.
The caller will use cpu_idle_wait something like this.

// Want to change idle handler
- Switch global idle handler to always present default_idle
- call cpu_idle_wait so that all cpus come out of idle for an instant
and stop using old idle pointer and start using default idle
- Change the idle handler to a new handler
- optional cpu_idle_wait if you want all cpus to start using the new
handler immediately.

May be the below 1s patch is safe bet for .24. But for .25, I would say
we just replace all complicated logic by simple dummy smp_call_function
and remove cpu_idle_state altogether.

Thanks,
Venki

>Signed-off-by: Steven Rostedt <srostedt@xxxxxxxxxx>
>---
> arch/x86/kernel/process_32.c |   11 +++++++++++
> arch/x86/kernel/process_64.c |   11 +++++++++++
> 2 files changed, 22 insertions(+)
>
>Index: linux-compile-i386.git/arch/x86/kernel/process_32.c
>===================================================================
>--- linux-compile-i386.git.orig/arch/x86/kernel/process_32.c	
>2008-01-09 14:09:36.000000000 -0500
>+++ linux-compile-i386.git/arch/x86/kernel/process_32.c	
>2008-01-09 14:09:45.000000000 -0500
>@@ -204,6 +204,10 @@ void cpu_idle(void)
> 	}
> }
> 
>+static void do_nothing(void *unused)
>+{
>+}
>+
> void cpu_idle_wait(void)
> {
> 	unsigned int cpu, this_cpu = get_cpu();
>@@ -228,6 +232,13 @@ void cpu_idle_wait(void)
> 				cpu_clear(cpu, map);
> 		}
> 		cpus_and(map, map, cpu_online_map);
>+		/*
>+		 * We waited 1 sec, if a CPU still did not call idle
>+		 * it may be because it is in idle and not waking up
>+		 * because it has nothing to do.
>+		 * Give all the remaining CPUS a kick.
>+		 */
>+		smp_call_function_mask(map, do_nothing, 0, 0);
> 	} while (!cpus_empty(map));
> 
> 	set_cpus_allowed(current, tmp);
>Index: linux-compile-i386.git/arch/x86/kernel/process_64.c
>===================================================================
>--- linux-compile-i386.git.orig/arch/x86/kernel/process_64.c	
>2008-01-09 14:09:36.000000000 -0500
>+++ linux-compile-i386.git/arch/x86/kernel/process_64.c	
>2008-01-09 15:17:20.000000000 -0500
>@@ -135,6 +135,10 @@ static void poll_idle (void)
> 	cpu_relax();
> }
> 
>+static void do_nothing(void *unused)
>+{
>+}
>+
> void cpu_idle_wait(void)
> {
> 	unsigned int cpu, this_cpu = get_cpu();
>@@ -160,6 +164,13 @@ void cpu_idle_wait(void)
> 				cpu_clear(cpu, map);
> 		}
> 		cpus_and(map, map, cpu_online_map);
>+		/*
>+		 * We waited 1 sec, if a CPU still did not call idle
>+		 * it may be because it is in idle and not waking up
>+		 * because it has nothing to do.
>+		 * Give all the remaining CPUS a kick.
>+		 */
>+		smp_call_function_mask(map, do_nothing, 0, 0);
> 	} while (!cpus_empty(map));
> 
> 	set_cpus_allowed(current, tmp);
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/