[PATCH] perf/x86/pebs: catch all PEBS overflow conditions

[Stephane Eranian]

This patch fixes an issue with the IRQ handler for the PMU when
PEBS events are combined with non-PEBS events such as the NMI
watchdog.

The setting of the OVF status register appears to be racy. In
a situation with 1 non-PEBS and multiple PEBS events, if the
non-PEBS counter overflows first, the handler may be entered
showing a non-PEBS counter overflowed and possibly also a PEBS
counter overflowed but without having bit 62 set. In that case the
current handler is considering the overflow of the PEBS counter as
general non-PEBS and this would result in a non-EXACT sample.

The fix is for PEBS to check if either bit 62 is set or if any of the
active PEBS counters bits are set in the OVF status. Either one is
a sign that there may be some samples to process via drain_pebs().

With this patch applied on top of peterz queue.tip perf/core, I do
not see non-EXACT samples for PEBS events on Haswell and later.

Signed-off-by: Stephane Eranian <eranian@xxxxxxxxxx>
---
 arch/x86/events/intel/core.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 87f52d6..e535894 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -1831,8 +1831,11 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 
 	/*
 	 * PEBS overflow sets bit 62 in the global status register
+	 * This is racy with overflows of non-PEBS counters, so we
+	 * status may not have bit 62 set yet pebs counters may have overflowed
 	 */
-	if (__test_and_clear_bit(62, (unsigned long *)&status)) {
+	if (__test_and_clear_bit(62, (unsigned long *)&status)
+	    || (status & cpuc->pebs_enabled)) {
 		handled++;
 		x86_pmu.drain_pebs(regs);
 		/*
@@ -1845,6 +1848,9 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)
 		 */
 		status &= ~cpuc->pebs_enabled;
 		status &= x86_pmu.intel_ctrl | GLOBAL_STATUS_TRACE_TOPAPMI;
+
+		/* make sure bit 62 is acked in any case */
+		orig_status |= 1ULL << 62;
 	}
 
 	/*
-- 
2.5.0