[PATCH 0/4] CXL 3.0 Performance Monitoring Unit support

From: Jonathan Cameron
Date: Fri Mar 03 2023 - 12:50:32 EST


There was some interest in CPMU support on the monthly sync call, so
time to revisit this. The general CXL driver interrupt handling question
has now been resolved, so this is pretty separable from everything else
going on in CXL land.

Since RFC v3:
- Simplified registration as no longer need the dynamic interrupt
discovery dance as upstream driver is just enabling lots of interrupts :)
- Various minor things (I forgot to take note).

Updated cover letter:

The CXL rev 3.0 specification introduces a CXL Performance Monitoring
Unit definition. CXL components may have any number of these blocks. The
definition is highly flexible, but that does bring complexity in the
driver.

One big question - how do we expose discoverable counters when we don't know
what they are? (just VID, Group ID and Mask)

A few things to note.

1) The QEMU model against which this was developed needs tidying up and
review for correctness. Latest tree at
https://gitlab.com/jic23/qemu branch cxl-2023-02-28
It's backed up behind other series that I plan to upstream first.
2) There are quite a lot of corner cases that will need working through
with variants of the model, or I'll have to design a pathological
set of CPMUs to hit all the corner cases in one go. So whilst I 'think'
the driver should be fine for what it supports, I may have missed
something.
3) I'm not sure it makes sense to hang this of the cxl/pci driver but
couldn't really figure out where else in the current structure we could
make it fit cleanly.
4) I'm not sure how to expose to user space the sets of events that may
be summed (given by a mask in the Counter Event Capabilities registers).
For now the driver advertises the individual events. Each individual
event may form part of multiple overlapping groups for example.
It may be a case of these allowed combinations only being discoverable
by requesting a combination and checking for errors on start.
5) Related to that, how do we deal with the discoverable nature of vendor
defined events? The CPMU is fully discoverable down to a VID / Group ID
and mask that sets which group elements can be used. I can't find any
similar drivers.
6) Driver location. In past perf maintainers have requested perf drivers
for PCI etc be under drivers/perf. That would require moving some
CXL headers to be more generally visible, but is certainly possible
if there is agreement between CXL and perf maintainers on the correct
location.
7) Documentation needs improving, but I didn't want to spend too much
time on that whilst we have so many open questions. I'll separately
raise the question about pmu->dev parenting which is mentioned in the
Docs patch introduction. (oops - haven't done that yet)

CXL rev 3.0 specification available from https://www.computeexpresslink.org

Based on mainline as of earlier today.

Jonathan Cameron (4):
cxl: Add function to count regblocks of a given type.
cxl/pci: Find and register CXL PMU devices
cxl: CXL Performance Monitoring Unit driver
docs: perf: Minimal introduction the the CXL PMU device and driver.

Documentation/admin-guide/perf/cxl.rst | 65 ++
Documentation/admin-guide/perf/index.rst | 1 +
drivers/cxl/Kconfig | 12 +
drivers/cxl/Makefile | 1 +
drivers/cxl/core/Makefile | 1 +
drivers/cxl/core/core.h | 3 +
drivers/cxl/core/cpmu.c | 69 ++
drivers/cxl/core/pci.c | 2 +-
drivers/cxl/core/port.c | 4 +-
drivers/cxl/core/regs.c | 50 +-
drivers/cxl/cpmu.c | 943 +++++++++++++++++++++++
drivers/cxl/cxl.h | 17 +-
drivers/cxl/cxlpci.h | 1 +
drivers/cxl/pci.c | 27 +-
14 files changed, 1188 insertions(+), 8 deletions(-)
create mode 100644 Documentation/admin-guide/perf/cxl.rst
create mode 100644 drivers/cxl/core/cpmu.c
create mode 100644 drivers/cxl/cpmu.c

--
2.37.2