[PATCH tip/core/rcu 0/40] SRCU callback parallelization for 4.12

[Paul E. McKenney]

Hello!

This series moves SRCU from its traditional single per-srcu_struct
callback queue to per-srcu_struct/per-CPU callback queues.  This involves
abstracting functionality from Tree RCU, which results in a large
conflict footprint, which in turn results in some otherwise unrelated
patches coming along for the ride.

1.	Maintain special bits at bottom of ->dynticks counter.
	This is for some upcoming MM work.  My intent was to hold
	it until that work was ready, but merge conflicts dictated
	otherwise.  If the MM work does not appear soonish, I will
	manually revert this patch.

2.	Make arch select smp_mb__after_unlock_lock() strength, which
	gets rid of an arch-specific #ifdef.

3.	Consolidate SRCU batch checking into rcu_all_batches_empty().

4.	Check for tardy grace-period activity in cleanup_srcu_struct().

5-7.	Semicolon inside RCU_TRACE() for various parts of RCU.

8-10.	Make various parts of RCU do deferred NOCB wakeups in order to
	prevent callback blockages, and thus hangs.

11.	Pull rcu_sched_qs_mask into rcu_dynticks structure in order to
	eliminate an isolated per-CPU variable.

12.	Pull rcu_qs_ctr into rcu_dynticks structure.

13.	Eliminate flavor scan in rcu_momentary_dyntick_idle() to
	reduce semi-common-case context-switch overhead.

14.	Place guard on rcu_all_qs() and rcu_note_context_switch()
	actions to reduce common-case scheduler-fastpath overhead.

15.	Default RCU_FANOUT_LEAF to 16 unless explicitly changed.

16.	Abstract multi-tail callback list handling for SRCU.

17.	Allow SRCU to access rcu_scheduler_active.

18.	Allow early boot use of synchronize_srcu(), though not yet
	mid-boot use.

19.	Add single-element dequeue functions to rcu_segcblist for
	debug use.

20.	Move rcu_seq_start() and friends to rcu.h for SRCU's benefit.

21.	Expedited wakeups need to be fully ordered.

22.	Fix warning in rcu_seq_end().

23.	Push srcu_advance_batches() fastpath into common case as a
	step towards callback parallelization.

24.	Move to state-based grace-period sequencing, also as a step
	towards callback parallelization.

25.	Add grace-period sequence numbers to SRCU.

26.	Use rcu_segcblist to track SRCU callbacks.

27.	Move combining-tree definitions for SRCU's benefit.

28.	Move rcu_init_levelspread() to rcu_tree_node.h for SRCU's benefit.

29.	Remove redundant levelcnt[] array from rcu_init_one().

30.	Move rcu_node traversal macros to rcu.h for SRCU's benefit.

31.	Make num_rcu_lvl[] array be external for SRCU's benefit.

32.	Fix bogus try_check_zero() comment.

33.	Improve rcu_seq grace-period-counter abstraction for SRCU's
	benefit.

34.	Allow a second bit in rcu_seq for SRCU state.

35.	Merge ->srcu_state into ->srcu_gp_seq to allow atomic updates.

36.	Provide crude control of expedited SRCU grace periods.

37.	Create a tiny SRCU for bloatwatch/tinification.

38.	Print Tiny SRCU reader statistics in rcutorture.

39.	Introduce CLASSIC_SRCU Kconfig option for those who do not
	wish to help debug Tree SRCU.

40.	Parallelize SRCU callback handling.

							Thanx, Paul

------------------------------------------------------------------------

 /kernel/rcu/rcu_segcblist.h                                     |  671 -----
 b/Documentation/RCU/Design/Data-Structures/Data-Structures.html |   36 
 b/arch/Kconfig                                                  |    3 
 b/arch/powerpc/Kconfig                                          |    1 
 b/include/linux/rcu_node_tree.h                                 |  105 
 b/include/linux/rcu_segcblist.h                                 |  720 +++++
 b/include/linux/rcupdate.h                                      |    6 
 b/include/linux/rcutiny.h                                       |   11 
 b/include/linux/srcu.h                                          |  112 
 b/include/linux/srcuclassic.h                                   |  101 
 b/include/linux/srcutiny.h                                      |   81 
 b/include/linux/srcutree.h                                      |  171 +
 b/init/Kconfig                                                  |   33 
 b/kernel/rcu/Makefile                                           |    6 
 b/kernel/rcu/rcu.h                                              |  165 +
 b/kernel/rcu/rcu_segcblist.h                                    |  671 +++++
 b/kernel/rcu/rcutorture.c                                       |   39 
 b/kernel/rcu/srcu.c                                             |  846 +++---
 b/kernel/rcu/srcutiny.c                                         |  215 +
 b/kernel/rcu/srcutree.c                                         | 1252 ++++++++--
 b/kernel/rcu/tiny.c                                             |   20 
 b/kernel/rcu/tiny_plugin.h                                      |   13 
 b/kernel/rcu/tree.c                                             |  650 +----
 b/kernel/rcu/tree.h                                             |  174 -
 b/kernel/rcu/tree_exp.h                                         |   25 
 b/kernel/rcu/tree_plugin.h                                      |   70 
 b/kernel/rcu/tree_trace.c                                       |   26 
 b/kernel/rcu/update.c                                           |   52 
 28 files changed, 4261 insertions(+), 2014 deletions(-)