Re: [PATCH -rt] Revert "net: use synchronize_rcu_expedited()"

[Josh Cartwright]

On Mon, Oct 26, 2015 at 05:44:22PM -0700, Paul E. McKenney wrote:
> On Mon, Oct 26, 2015 at 02:14:55PM -0500, Josh Cartwright wrote:
> > This reverts commit be3fc413da9eb17cce0991f214ab019d16c88c41.
> > 
> > While the use of synchronize_rcu_expedited() might make
> > synchronize_net() "faster", it does so at significant cost on RT
> > systems, as expediting a grace period forcibly preempts any
> > high-priority RT tasks (via the stop_machine() mechanism).
> > 
> > Without be3fc413da9e reverted, we can observe a latency spike up to 30us
> > with cyclictest by rapidly unplugging/reestablishing an ethernet link.
[..]
> 
> Hmmm...  If I remember correctly, using expedited here resulted
> in impressive performance improvements in some important cases.
> Perhaps things have changed (I must defer to Eric), but if not, how
> about something like this instead?
> 
> 	if (rtnl_is_locked() && !IS_ENABLED(CONFIG_PREMPT_RT_FULL))
> 		synchronize_rcu_expedited();
> 	else
> 		synchronize_rcu();
> 
> Alternatively, a boot-time option could be used:
> 
> 	if (rtnl_is_locked() && !some_rt_boot_parameter)
> 		synchronize_rcu_expedited();
> 	else
> 		synchronize_rcu();
> 
> I believe that the first alternative is better because it does the right
> thing without user intervention.  The second would be preferred should
> distros want to offer full RT by default, but I am guessing thta most
> distros would be reluctant to do this for some time to come.
> 
> Either way, these approaches have the advantage of giving RT users the
> latency they need, even in the face of networking configuration changes,
> while giving non-RT users the required performance of the networking
> configuration changes themselves.

Okay, yes, I like the first suggestion better as well, I've included a
patch below that does just that.  I hope you don't mind me turning it
into a Suggested-by :).

Thanks for taking a look!
  Josh

-- 8< --
From: Josh Cartwright <joshc@xxxxxx>
Subject: [PATCH] net: make synchronize_rcu_expedited() conditional on RT_FULL

While the use of synchronize_rcu_expedited() might make
synchronize_net() "faster", it does so at significant cost on RT
systems, as expediting a grace period forcibly preempts any
high-priority RT tasks (via the stop_machine() mechanism).

Without this change, we can observe a latency spike up to 30us with
cyclictest by rapidly unplugging/reestablishing an ethernet link.

Cc: Eric Dumazet <eric.dumazet@xxxxxxxxx>
Cc: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
Cc: David S. Miller <davem@xxxxxxxxxxxxx>
Suggested-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
Signed-off-by: Josh Cartwright <joshc@xxxxxx>
---
 net/core/dev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/core/dev.c b/net/core/dev.c
index f8c23de..16fbef8 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -6969,7 +6969,7 @@ EXPORT_SYMBOL(free_netdev);
 void synchronize_net(void)
 {
 	might_sleep();
-	if (rtnl_is_locked())
+	if (rtnl_is_locked() && !IS_ENABLED(CONFIG_PREEMPT_RT_FULL))
 		synchronize_rcu_expedited();
 	else
 		synchronize_rcu();
-- 
2.5.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/