Re: [PATCH v9 03/10] perf/x86: Add constraint to create guest LBR event without hw counter

From: Xu, Like
Date: Fri Apr 17 2020 - 04:40:18 EST

Hi Peter,

On 2020/4/10 11:03, Xu, Like wrote:
Hi Peter,

First of all, thanks for your comments!

On 2020/4/10 0:37, Peter Zijlstra wrote:
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 3bb738f5a472..e919187a0751 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -74,7 +74,8 @@ u64 x86_perf_event_update(struct perf_event *event)
ÂÂÂÂÂ int idx = hwc->idx;
ÂÂÂÂÂ u64 delta;
 - if (idx == INTEL_PMC_IDX_FIXED_BTS)
+ÂÂÂ if ((idx == INTEL_PMC_IDX_FIXED_BTS) ||
+ÂÂÂÂÂÂÂ (idx == INTEL_PMC_IDX_FIXED_VLBR))
ÂÂÂÂÂÂÂÂÂ return 0;
 Â /*
@@ -1102,7 +1103,8 @@ static inline void x86_assign_hw_event(struct perf_event *event,
ÂÂÂÂÂ hwc->last_cpu = smp_processor_id();
ÂÂÂÂÂ hwc->last_tag = ++cpuc->tags[i];
 - if (hwc->idx == INTEL_PMC_IDX_FIXED_BTS) {
+ÂÂÂ if ((hwc->idx == INTEL_PMC_IDX_FIXED_BTS) ||
+ÂÂÂÂÂÂÂ (hwc->idx == INTEL_PMC_IDX_FIXED_VLBR)) {
ÂÂÂÂÂÂÂÂÂ hwc->config_base = 0;
ÂÂÂÂÂÂÂÂÂ hwc->event_baseÂÂÂ = 0;
ÂÂÂÂÂ } else if (hwc->idx >= INTEL_PMC_IDX_FIXED) {
@@ -1233,7 +1235,8 @@ int x86_perf_event_set_period(struct perf_event *event)
ÂÂÂÂÂ s64 period = hwc->sample_period;
ÂÂÂÂÂ int ret = 0, idx = hwc->idx;
 - if (idx == INTEL_PMC_IDX_FIXED_BTS)
+ÂÂÂ if ((idx == INTEL_PMC_IDX_FIXED_BTS) ||
+ÂÂÂÂÂÂÂ (idx == INTEL_PMC_IDX_FIXED_VLBR))
ÂÂÂÂÂÂÂÂÂ return 0;
 Â /*
That seems unfortunate; can that be >= INTEL_PMC_IDX_FIXED_BTS ? If so,
that probably wants a comment with the definitions.

Or otherwise check for !hwc->event_base. That should be 0 for both these
things.
Yes, the !hwc->event_base looks good to me.

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 3be51aa06e67..901c82032f4a 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -2157,6 +2157,9 @@ static void intel_pmu_disable_event(struct perf_event *event)
ÂÂÂÂÂÂÂÂÂ return;
ÂÂÂÂÂ }
 + if (unlikely(hwc->idx == INTEL_PMC_IDX_FIXED_VLBR))
+ÂÂÂÂÂÂÂ return;
+
Please check code-gen to see if you can cut down on brancher here;
there's 4 cases:

 - vlbr
 - bts
 - fixed
 - gp

perhaps you can write it like so:

(also see https://lkml.kernel.org/r/20190828090217.GN2386@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx )

static void intel_pmu_enable_event(struct perf_event *event)
{
ÂÂÂÂ...
ÂÂÂÂint idx = hwx->idx;

ÂÂÂÂif (idx < INTEL_PMC_IDX_FIXED) {
ÂÂÂÂÂÂÂ intel_set_masks(event, idx);
ÂÂÂÂÂÂÂ __x86_pmu_enable_event(hwc, ARCH_PERFMON_EVENTSEL_ENABLE);
ÂÂÂÂ} else if (idx < INTEL_PMC_IDX_FIXED_BTS) {
ÂÂÂÂÂÂÂ intel_set_masks(event, idx);
ÂÂÂÂÂÂÂ intel_pmu_enable_fixed(event);
ÂÂÂÂ} else if (idx == INTEL_PMC_IDX_FIXED_BTS) {
ÂÂÂÂÂÂÂ intel_pmu_enable_bts(hwc->config);
ÂÂÂÂ}

ÂÂÂÂ/* nothing for INTEL_PMC_IDX_FIXED_VLBR */
}

That should sort the branches in order of: gp,fixed,bts,vlbr

Note the current order is: bts, pebs, fixed, gp.

Sure, let me try to refactor it in this way.

ÂÂÂÂÂ cpuc->intel_ctrl_guest_mask &= ~(1ull << hwc->idx);
ÂÂÂÂÂ cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx);
ÂÂÂÂÂ cpuc->intel_cp_status &= ~(1ull << hwc->idx);
@@ -2241,6 +2244,9 @@ static void intel_pmu_enable_event(struct perf_event *event)
ÂÂÂÂÂÂÂÂÂ return;
ÂÂÂÂÂ }
 + if (unlikely(hwc->idx == INTEL_PMC_IDX_FIXED_VLBR))
+ÂÂÂÂÂÂÂ return;
+
ÂÂÂÂÂ if (event->attr.exclude_host)
ÂÂÂÂÂÂÂÂÂ cpuc->intel_ctrl_guest_mask |= (1ull << hwc->idx);
ÂÂÂÂÂ if (event->attr.exclude_guest)
idem.
idem.

@@ -2595,6 +2601,15 @@ intel_bts_constraints(struct perf_event *event)
ÂÂÂÂÂ return NULL;
 }
 +static struct event_constraint *
+intel_guest_event_constraints(struct perf_event *event)
+{
+ÂÂÂ if (unlikely(is_guest_lbr_event(event)))
+ÂÂÂÂÂÂÂ return &guest_lbr_constraint;
+
+ÂÂÂ return NULL;
+}
This is a mis-nomer, it isn't just any guest_event

Sure, I'll rename it to intel_guest_lbr_event_constraints()
instead of using it as a unified interface to get all of guest event constraints.


+
 static int intel_alt_er(int idx, u64 config)
 {
ÂÂÂÂÂ int alt_idx = idx;
@@ -2785,6 +2800,10 @@ __intel_get_event_constraints(struct cpu_hw_events *cpuc, int idx,
 {
ÂÂÂÂÂ struct event_constraint *c;
 + c = intel_guest_event_constraints(event);
+ÂÂÂ if (c)
+ÂÂÂÂÂÂÂ return c;
+
ÂÂÂÂÂ c = intel_bts_constraints(event);
ÂÂÂÂÂ if (c)
ÂÂÂÂÂÂÂÂÂ return c;
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index 1025bc6eb04f..9a62264a3068 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -969,6 +969,20 @@ static inline bool intel_pmu_has_bts(struct perf_event *event)
ÂÂÂÂÂ return intel_pmu_has_bts_period(event, hwc->sample_period);
 }
 +static inline bool is_guest_event(struct perf_event *event)
+{
+ÂÂÂ if (event->attr.exclude_host && is_kernel_event(event))
+ÂÂÂÂÂÂÂ return true;
+ÂÂÂ return false;
+}
I don't like this one, what if another in-kernel users generates an
event with exclude_host set ?
Thanks for the clear attitude.

How about:
- remove the is_guest_event() to avoid potential misuse;
- move all checks into is_guest_lbr_event() and make it dedicated:

static inline bool is_guest_lbr_event(struct perf_event *event)
{
ÂÂÂ if (is_kernel_event(event) &&
ÂÂÂ ÂÂÂ event->attr.exclude_host && needs_branch_stack(event))
ÂÂÂ ÂÂÂ return true;
ÂÂÂ return false;
}

In this case, it's safe to generate an event with exclude_host set
and also use LBR to count guest or nothing for other in-kernel users
because the intel_guest_lbr_event_constraints() makes LBR exclusive.

For this generic usage, I may rename:
- is_guest_lbr_event() to is_lbr_no_counter_event();
- intel_guest_lbr_event_constraints() to intel_lbr_no_counter_event_constraints();

Is this acceptable to youï
If there is anything needs to be improved, please let me know.
Do you have any preference for this ?

If you have more comments for the general idea or code details, please let me know.
For example, you may take a look at the interface named intel_pmu_create_lbr_event()
in the "[PATCH v9 07/10] KVM: x86/pmu: Add LBR feature emulation via guest LBR event".

If not, I'll spin the next version based on your current feedback.

Thanks,
Like Xu

@@ -989,6 +1003,7 @@ void release_ds_buffers(void);
 void reserve_ds_buffers(void);
  extern struct event_constraint bts_constraint;
+extern struct event_constraint guest_lbr_constraint;
  void intel_pmu_enable_bts(u64 config);
 diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index e018a1cf604c..674130aca75a 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -181,9 +181,19 @@ struct x86_pmu_capability {
 #define GLOBAL_STATUS_UNC_OVF BIT_ULL(61)
 #define GLOBAL_STATUS_ASIF BIT_ULL(60)
 #define GLOBAL_STATUS_COUNTERS_FROZEN BIT_ULL(59)
-#define GLOBAL_STATUS_LBRS_FROZENÂÂÂÂÂÂÂÂÂÂÂ BIT_ULL(58)
+#define GLOBAL_STATUS_LBRS_FROZEN_BITÂÂÂÂÂÂÂÂÂÂÂ 58
+#define GLOBAL_STATUS_LBRS_FROZEN BIT_ULL(GLOBAL_STATUS_LBRS_FROZEN_BIT)
 #define GLOBAL_STATUS_TRACE_TOPAPMI BIT_ULL(55)
 +/*
+ * We model guest LBR event tracing as another fixed-mode PMC like BTS.
+ *
+ * We choose bit 58 (LBRS_FROZEN_BIT) which is used to indicate that the LBR
+ * stack is frozen on a hardware PMI request in the PERF_GLOBAL_STATUS msr,
+ * and the 59th PMC counter (if any) is not supposed to use it as well.
Is this saying that STATUS.58 should never be set? I don't really
understand the language.
My fault, and let me make it more clearly:

We choose bit 58 because it's used to indicate LBR stack frozen state
not like other overflow conditions in the PERF_GLOBAL_STATUS msr,
and it will not be used for any actual fixed events.


+ */
+#define INTEL_PMC_IDX_FIXED_VLBR GLOBAL_STATUS_LBRS_FROZEN_BIT
+
 /*
ÂÂ * Adaptive PEBS v4
ÂÂ */