[Patch v6 00/20] perf/x86: Fix multiple LBR sampling issues

[Dapeng Mi]

This patch series fixes several x86 LBR sampling issues discovered during
review and debugging. Most of them are related to BPF LBR snapshotting
support.

The series originally targeted the BPF stack out-of-bounds access issue,
but additional problems were found during debugging and code review,
including:
- PMU and LBR being re-enabled unconditionally at BPF snapshotting exit,
  even when they had already been disabled before BPF snapshotting on both
  AMD and Intel.
- LBR not being fully frozen in the PMI handler on Intel, which allows
  branches in the PMI path to pollute the BPF LBR snapshot.
- BPF snapshotting overwriting the cached LBR stack of PEBS LBR events on
  Intel.
- PEBS LBR events overwriting the LBR stack of non-PEBS LBR events on
  Intel when both overflow in the same PMI.
- LBR counter events reporting zero counters on Intel when multiple
  LBR counter events overflow in the same PMI.


Patch layout:
- Patch 01: Add a sanity check to ensure the LBR stack does not exceed
  MAX_LBR_ENTRIES.
- Patch 02: Validate that hardware LBR data has actually been stored in
  the XSAVE area before parsing it.
- Patch 03-04: Fix the potential LBR stack out-of-bounds access issue on
  AMD and Intel separately.
- Patch 05-08: Fix the incorrect PMU and LBR re-enabling issue
  in perf_snapshot_branch_stack() for AMD and Intel separately.
- Patch 09: Freeze LBR throughout the PMI processing period and ensure
  all LBR events observe a consistent LBR stack in the PMI handler.
- Patch 10: Add counted bounds tracking for perf_branch_stack{}.
- Patch 11-12: Add a dedicated LBR stack for BPF LBR snapshotting to avoid
  overwriting the default LBR stack of other LBR events.
- Patch 13: Ensure perf_snapshot_branch_stack() snapshots the live
  hardware LBR data instead of the cached PEBS LBR stack.
- Patch 14: Refresh LBR stack in PMI only when LBR users are active for
  AMD.
- Patch 15: Prevent PEBS LBR data from overwriting non-PEBS LBR data when
  both overflow in the same PMI.
- Patch 16: Remove dead lbr_pebs_users and relative code.
- Patch 17-18: Ensure LBR counters are not dropped when multiple LBR
  counter events overflow in the same PMI.
- Patch 19: Clamp the parsed PEBS LBR stack depth to x86_pmu.lbr_nr to
  avoid LBR stack out-of-bounds access.
- Patch 20: Fix the shared LBR stack depth truncation issue.


Changes in v6:
- Patch 01: Move LBR stack depth validation into common
  init_hw_perf_events(). (Peter)
- Patch 10: New patch, add __counted_by() tracking for flexible array
  entries[] of perf_branch_stack{}. (Peter)
- Patch 14: Refresh LBR stack only when LBR users are active for AMD.
  (Peter)
- Patch 15: Refactor code to ensure no duplicated LBR stack reading.
  (Peter)
- Patch 16: New patch, remove dead lbr_pebs_users.

Changes in v5:
- Patch 02: Clear xsave->header.xfeatures before calling the xsaves
  instructuion to read LBR states.
- Patch 05-18: New patches to fix newly discovered issues described above.

Changes in v4:
- Patch 01: Disable LBR if parsed x86_pmu.lbr_nr exceeds
  MAX_LBR_ENTRIES (Sashiko).
- Patch 05: New patch, which always snapshots live hardware LBR stack
  for BPF path (Sashiko).

Changes in v3:
- Patch 01: A new patch to ensure x86_pmu.lbr_nr not to exclude
  MAX_LBR_ENTRIES (Internal Sashiko).
- Patch 03: Clean cached LBR stack when reseting hardware LBR registers,
  which prevents stale LBR stack leakage.
- Patch 04: Don't reset cached LBR stack if all LBR events are PEBS
  events. This ensures PEBS captured LBR stack isn't cleared (Sashiko).

Changes in v2:
- Include a fix for Intel LBR.
- Add Tested-by from Dapeng to the Intel fix.
  https://lore.kernel.org/all/eb6c7bda-0dcf-4fac-ad69-9f5b24abc21b@xxxxxxxxxxxxxxx/

History:
v5: https://lore.kernel.org/all/20261029043914.1395147-1-dapeng1.mi@xxxxxxxxxxxxxxx/
v4: https://lore.kernel.org/all/20261021050711.4043703-1-dapeng1.mi@xxxxxxxxxxxxxxx/
v3: https://lore.kernel.org/all/20261019070836.2972403-1-dapeng1.mi@xxxxxxxxxxxxxxx/
v2: https://lore.kernel.org/all/cover.1792064123.git.sandipan.das@xxxxxxx/
v1: https://lore.kernel.org/all/20261007181002.4005262-1-sandipan.das@xxxxxxx/

Dapeng Mi (19):
  perf/x86: Disable LBR if x86_pmu.lbr_nr exceeds MAX_LBR_ENTRIES
  perf/x86/intel: Validate LBR XSAVE state before updating LBR stack
  perf/x86/intel/lbr: Limit branch snapshots to valid entries
  perf/x86: Add lbr_enabled to track LBR state
  perf/x86/amd: Avoid re-enabling PMU and LBR for BPF when disabled
  perf/x86/intel: Add helper to enable PMU without LBR
  perf/x86/intel: Avoid re-enabling PMU and LBR for BPF when disabled
  perf/x86/intel: Keep LBR frozen throughout PMI handling
  perf: Add counted-bounds tracking for perf_branch_stack
  perf/x86/intel: Allow LBR read helpers to operate on explicit stacks
  perf/x86/intel: Use a dedicated LBR stack for BPF snapshotting
  perf/x86/intel: Always snapshot live hardware LBR state for BPF
  perf/x86/amd: Refresh LBR stack in PMI only when LBR users are active
  perf/x86/intel: Refresh LBR stack for non-PEBS LBR events when needed
  perf/x86/intel: Remove dead lbr_pebs_users
  perf/x86/intel: Clear raw LBR counters in the LBR stack
  perf/x86/intel: Refill LBR stack for LBR counter events in PMI handler
  perf/x86/intel: Pass LBR stack size into store_lbr()
  perf/core: Don't overwrite shared LBR stack depth

Sandipan Das (1):
  perf/x86/amd/lbr: Limit branch snapshots to valid entries

 arch/powerpc/perf/core-book3s.c |  43 +++++---
 arch/x86/events/amd/brs.c       |   7 +-
 arch/x86/events/amd/core.c      |  21 ++--
 arch/x86/events/amd/lbr.c       |  48 +++++----
 arch/x86/events/core.c          |   8 +-
 arch/x86/events/intel/core.c    | 130 ++++++++++++++++++++----
 arch/x86/events/intel/ds.c      |  20 ++--
 arch/x86/events/intel/lbr.c     | 170 +++++++++++++++++++-------------
 arch/x86/events/perf_event.h    |  45 +++++++--
 drivers/perf/arm_brbe.c         |   5 +
 include/linux/perf_event.h      |  15 ++-
 kernel/events/core.c            |   8 +-
 12 files changed, 365 insertions(+), 155 deletions(-)


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.34.1