Re: [PATCH RFC] v5 expedited "big hammer" RCU grace periods

From: Ingo Molnar
Date: Mon May 18 2009 - 03:57:57 EST



* Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx> wrote:

> +void sched_expedited_wake(void *unused)
> +{
> + mutex_lock(&__get_cpu_var(sched_expedited_done_mutex));
> + if (__get_cpu_var(sched_expedited_done_qs) ==
> + SCHED_EXPEDITED_QS_DONE_QS) {
> + __get_cpu_var(sched_expedited_done_qs) =
> + SCHED_EXPEDITED_QS_NEED_QS;
> + wake_up(&__get_cpu_var(sched_expedited_qs_wq));
> + }
> + mutex_unlock(&__get_cpu_var(sched_expedited_done_mutex));
> +}

( hm, IPI handlers are supposed to be atomic. )

> +/*
> + * Kernel thread that processes synchronize_sched_expedited() requests.
> + * This is implemented as a separate kernel thread to avoid the need
> + * to mess with other tasks' cpumasks.
> + */
> +static int krcu_sched_expedited(void *arg)
> +{
> + int cpu;
> + int mycpu;
> + int nwait;
> +
> + do {
> + wait_event_interruptible(need_sched_expedited_wq,
> + need_sched_expedited);
> + smp_mb(); /* In case we didn't sleep. */
> + if (!need_sched_expedited)
> + continue;
> + need_sched_expedited = 0;
> + get_online_cpus();
> + preempt_disable();
> + mycpu = smp_processor_id();
> + smp_call_function(sched_expedited_wake, NULL, 1);
> + preempt_enable();

i might be missing something fundamental here, but why not just have
per CPU helper threads, all on the same waitqueue, and wake them up
via a single wake_up() call? That would remove the SMP cross call
(wakeups do immediate cross-calls already).

Even more - we already have a per-CPU, high RT priority helper
thread that could be reused: the per CPU migration threads. Couldnt
we queue these requests to them? RCU is arguably closely related to
scheduling so there's no layering violation IMO.

There's already a struct migration_req machinery that performs
something quite similar. (do work on behalf of another task, on a
specific CPU, and then signal completion)

Also, per CPU workqueues have similar features as well.

Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/