[patch 00/39] hrtimer/tick: Optimizations, cleanups and solutions for various issues

[Thomas Gleixner]

When I returned from my break I got offended by a pile of patches
which kill the patient with the cure.

The issues at hand:

    - NOHZ: Get rid of the softirq invocation

    - hrtimer: Use the active_bases field in order to avoid evaluating
       	       inactive bases

    - hrtimer: Cache footprint issues

Aside of that Peter and I were discussing for a long time to get rid
of the hrtimer softirq.

After staring at all of it for quite some time it occured to me that
all issues are related in one way or the other. So I sat down and
reworked the code in various ways:

   - Reduce the data size, so the hrtimer clock bases can be made
     cache line aligned.

   - Consolidate everything on the high resolution timer
     implementation and get rid of dubious optimizations for the non
     highres case which bloat code and data.

   - Implement the active_bases mechanism proper and avoid touching
     inactive hrtimer clock bases. This includes a conditional update
     mechanism for the hrtimer clock offsets to update them only when
     they changed, which they do seldom enough instead of polluting 4
     cache lines in every tick/hrtimer interrupt.

   - Get rid of the softirq deferment and simply enforce a hrtimer
     interrupt when the timer was already expired. This allows to
     remove the ugly __hrtimer_start_range_ns() interface and to
     cleanup the usage sites (sched/perf). As a consequence this also
     gets rid of the forward loops in the tick nohz code.

   - Analogous to the hrtimer enforcement, force a tick interrupt for
     NOHZ non highres systems when the forwarding code tries to fire
     an expired tick. This allows to get rid of the softirq invocation
     in the NOHZ code.

   - A cleanup of the code which evaluates the next timer event: Use
     nsec based calculations instead of the jiffy magic. That makes it
     actually readable by some definition of readable.

   - While doing the above I had to audit quite some usage sites of
     various hrtimer interfaces, which revealed some entertaining
     bugs. The fixes have been posted in a seperate series
     already. Some other bogosities have been removed as part of this
     series.

The total change size of this overhaul is:

   37 files changed, 515 insertions(+), 794 deletions(-)

The resulting text size of hrtimers.o shrinks in the range of 8-10%
depending on the architecture.

The cache foot print of the hrtimer per cpu data shrinks as well.

    	  x8684	     i386   ARM	    ARM64   power64
Before:	  328	     248    280	    328	    328		Bytes
	    6	       4      5	      6	      6		cache lines (64byte)

After:	  320	     192    192	    320	    320		Bytes
	    5	       3      3	      5	      5		cache lines (64byte)

Note, that the new code avoids to touch the inactive clock bases which
are now cache line aligned and therefor reduces the cache foot print
in normal usage scenarios significantly.

I did some perf measurements on an isolated core running 

  - hrtimer centric workloads
  - idle scenarios with periodic wakeups of various length

The patches reduce the number of instructions executed during the test
runs between 2.5 and 6% depending on the scenario and the cache misses
between 3 and 8%.

For your convenience this series is also available at:

   git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/wip

   Note: The branch is temporary and not meant to base other work on it.

Thanks,

	tglx
---
 arch/x86/kernel/cpu/perf_event_intel_rapl.c   |    5 
 arch/x86/kernel/cpu/perf_event_intel_uncore.c |    5 
 drivers/power/reset/ltc2952-poweroff.c        |   18 
 drivers/staging/ozwpan/ozpd.c                 |    8 
 include/linux/alarmtimer.h                    |    4 
 include/linux/hrtimer.h                       |  101 ++---
 include/linux/interrupt.h                     |    7 
 include/linux/rcupdate.h                      |    6 
 include/linux/rcutree.h                       |    2 
 include/linux/timekeeper_internal.h           |    2 
 include/linux/timer.h                         |    7 
 include/linux/timerqueue.h                    |    8 
 include/trace/events/irq.h                    |    1 
 kernel/events/core.c                          |    9 
 kernel/futex.c                                |    5 
 kernel/locking/rtmutex.c                      |    5 
 kernel/rcu/tree_plugin.h                      |   14 
 kernel/sched/core.c                           |   28 -
 kernel/sched/deadline.c                       |   12 
 kernel/sched/fair.c                           |    2 
 kernel/softirq.c                              |    2 
 kernel/time/alarmtimer.c                      |   17 
 kernel/time/hrtimer.c                         |  525 +++++++++-----------------
 kernel/time/posix-timers.c                    |   17 
 kernel/time/tick-broadcast-hrtimer.c          |    8 
 kernel/time/tick-internal.h                   |    2 
 kernel/time/tick-sched.c                      |  288 +++++---------
 kernel/time/tick-sched.h                      |    2 
 kernel/time/timekeeping.c                     |   55 --
 kernel/time/timekeeping.h                     |   10 
 kernel/time/timer.c                           |   79 +--
 kernel/time/timer_list.c                      |   14 
 lib/timerqueue.c                              |   10 
 net/core/pktgen.c                             |    2 
 net/sched/sch_api.c                           |    5 
 sound/core/hrtimer.c                          |    9 
 sound/drivers/pcsp/pcsp.c                     |   15 
 37 files changed, 515 insertions(+), 794 deletions(-)



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/