[PATCH RFC srcu] Expedite srcu_schedule_cbs_snp() callback invocation

From: Paul E. McKenney
Date: Tue Apr 18 2017 - 13:34:04 EST


Although Tree SRCU does reduce delays when there is at least one
synchronize_srcu_expedited() invocation pending, srcu_schedule_cbs_snp()
still waits for SRCU_INTERVAL before invoking callbacks. Since
synchronize_srcu_expedited() now posts a callback and waits for
that callback to do a wakeup, this destroys the expedited nature of
synchronize_srcu_expedited().

This commit therefore invokes callbacks immediately at the end of the
grace period when there is at least one synchronize_srcu_expedited()
invocation pending.

Reported-by: Marc Zyngier <marc.zyngier@xxxxxxx>
Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>

diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index 1e54f91c2ff7..744ffdb79d23 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -447,7 +447,8 @@ static void srcu_schedule_cbs_snp(struct srcu_struct *sp, struct srcu_node *snp)
int cpu;

for (cpu = snp->grplo; cpu <= snp->grphi; cpu++)
- srcu_schedule_cbs_sdp(per_cpu_ptr(sp->sda, cpu), SRCU_INTERVAL);
+ srcu_schedule_cbs_sdp(per_cpu_ptr(sp->sda, cpu),
+ atomic_read(&sp->srcu_exp_cnt) ? 0 : SRCU_INTERVAL);
}

/*