[PATCH 0/5] perf: add PMU driver for the MIPS Coherence Manager 3 performance counters

[Vladimir Kondratiev]

The Coherence Manager 3 (CM3) of the MIPS I6500 multiprocessing system
is the point where the requests of all cores, I/O coherence units and
the hardware prefetcher are serialized before reaching the shared L2
cache and the memory interface. It carries a small set of performance
counters which observe this traffic: two programmable 32-bit event
counters and a 32-bit cycle counter, programmed through the GCR global
debug block (GCR_DB_PC_*).

The counters can count requests serialized by the CM, requests received
by an IOCU, requests issued on the memory interface, AXI handshake
cycles on the memory and IOCU buses, and PDTrace statistics. Each event
is refined by a 64-bit qualifier made of groups of bits (originating
port, thread, command, cacheability, L1/L2 line state, eviction, bank,
scheduler, ...): a request is counted when it matches a set bit in
every group, so the same event can be narrowed down to, for example,
L2 misses caused by coherent reads from core 1.

This is the only visibility into the L2 and the memory interface that
the I6500 offers; the per-core counters stop at the L1. It is the
natural place to attribute memory bandwidth to cores and IOCUs, or to
measure the L2 hit rate of a workload, on SoCs such as the Mobileye
EyeQ6H.

This series adds an uncore perf driver for these counters.

Design notes
------------

The driver binds to the existing coherency-manager device tree node
("mti,mips-cm" / "mobileye,eyeq6-cm") and reuses the GCR mapping
already set up by mips_cm_probe(), so no new reg property is needed:
the GCR base is discovered from CP0 CMGCRBase as for everything else in
the CM. The register definitions go into asm/mips-cm.h next to the
other GCRs, following the existing accessor scheme.

Events are described by two fields: 'event' (config:0-8) is the number
written to GCR_DB_PC_EVENT, with 0x100 as a pseudo event selecting the
cycle counter, and 'qual' (config1:0-63) is the raw qualifier. A
qualifier of 0 picks a per-event default counting every occurrence;
this matters for the I/O and AXI events, whose qualifiers contain
encoded values (IOCU number, AXI channel) rather than masks, so "all
ones" is not a sensible default there. Named events cover the common
combinations and, since they are plain event=/qual= strings, serve as
templates for custom qualifiers.

A multi-cluster system has one CM3, and thus one set of counters, per
cluster. The number of clusters comes from GCR_CONFIG and one PMU is
registered per cluster, named mips_cm3_pmu_<cluster>. A cluster need
not have CPUs, online or at all, for its CM to be busy: its L2 and
memory interface may serve the I/O devices behind its IOCUs while its
cores are absent or powered down. The counters of a cluster are thus
accessed through the GCR redirect block (GCR_Cx_OTHER with BLOCK set
to the debug block) from whichever CPU the PMU runs on, which may be in
any cluster. The one thing required is that the CM of the counted
cluster is powered, which the driver checks in the CPC (the CM "core"
STAT_CONF sequencer state) when an event is opened; the counters of a
cluster that is powered down at probe time are initialized on first
use, and events on it fail with -ENODEV until then. The counters are
shared by the whole cluster: events are system-wide only, pinned to one
CPU which follows CPU hotplug, and sampling, task events and the
exclude_* attributes are rejected.

Overflow is where the SoC integration leaks in. The CM3 has a
so_cm_perf_cnt_int output, but whether it is wired to a GIC input is
the integrator's choice and is not discoverable. The binding therefore
gains an optional 'interrupts' property: when present the driver uses
it to extend the 32-bit counters, otherwise it polls them every 250 ms,
which is comfortably below the time the fastest event needs to wrap
at the CM3 clock rates. Both paths yield 64-bit counts. No DTS change
is included because the EyeQ6H routing of that signal is not documented
on my side; the driver works in polling mode without it.

The interrupt can only serve the cluster the CPUs live in, and only
when they all live in that one cluster. Each cluster's CM drives its
own interrupt output into its own GIC, and the GIC driver routes a
shared interrupt to the cluster of its affinity CPU, so a single
'interrupts' entry cannot serve several clusters at once. The PMUs of
the other clusters, and all PMUs when CPUs are spread over clusters,
are polled.

The driver can be built as a module. That requires exporting
mips_gcr_base, mips_cpc_base, mips_cm_{lock,unlock}_other() and, less
obviously, mips_cm_is64, since the CPS accessor macros consult it on
every 64-bit GCR access. All are exported _GPL.

Patches
-------

Patch 1 adds the GCR_DB_PC_* register and field definitions, with
local and redirect accessors.

Patch 2 exports the CM and CPC symbols needed by a modular user of the
GCR accessors.

Patch 3 extends the mti,mips-cm binding with the optional performance
counter interrupt.

Patch 4 is the driver itself together with its admin-guide document,
which describes the perf interface and transcribes the event numbers
and qualifier layouts so that users do not need the TRM to build a
qualifier.

Patch 5 enables the driver (as a module) in eyeq6_defconfig.

Testing
-------

Built with eyeq6_defconfig with the driver both built-in and as a
module; every patch of the series builds on its own, and the binding
and the document pass their respective checkers.

Run on the EyeQ6H board with 2 clusters, loaded as module in polling mode.

[   66.128375] mips_cm3_pmu coherency-manager: mips_cm3_pmu_0: 2 counters + cycles, polled
[   66.129617] mips_cm3_pmu coherency-manager: mips_cm3_pmu_1: 2 counters + cycles, polled

dev / # perf stat -a -e mips_cm3_pmu_0/cm_requests_l2_hit/,mips_cm3_pmu_0/cm_requests_l2_miss/,mips_cm3_p
mu_1/cm_requests_l2_hit/,mips_cm3_pmu_1/cm_requests_l2_miss/,mips_cm3_pmu_0/cycles/,mips_cm3_pmu_1/cycles
/ sleep 1

 Performance counter stats for 'system wide':

            505821      mips_cm3_pmu_0/cm_requests_l2_hit/                                   
             49083      mips_cm3_pmu_0/cm_requests_l2_miss/                                   
            184893      mips_cm3_pmu_1/cm_requests_l2_hit/                                   
             37760      mips_cm3_pmu_1/cm_requests_l2_miss/                                   
        1807514589      mips_cm3_pmu_0/cycles/                                      
        1807491360      mips_cm3_pmu_1/cycles/                                      

       1.005857149 seconds time elapsed

Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev@xxxxxxxxxxxx>
---
Vladimir Kondratiev (5):
      MIPS: CM: add CM3 performance counter GCR definitions
      MIPS: CM: export the symbols needed to access the GCRs from a module
      dt-bindings: mips: mti,mips-cm: add performance counter interrupt
      perf: add MIPS Coherence Manager 3 (CM3) PMU driver
      MIPS: eyeq6_defconfig: enable the CM3 PMU

 Documentation/admin-guide/perf/index.rst           |   1 +
 Documentation/admin-guide/perf/mips_cm3_pmu.rst    | 337 +++++++
 .../devicetree/bindings/mips/mti,mips-cm.yaml      |   6 +
 arch/mips/configs/eyeq6_defconfig                  |   2 +
 arch/mips/include/asm/mips-cm.h                    |  51 +-
 arch/mips/kernel/mips-cm.c                         |   5 +
 arch/mips/kernel/mips-cpc.c                        |   2 +
 drivers/perf/Kconfig                               |  12 +
 drivers/perf/Makefile                              |   1 +
 drivers/perf/mips_cm3_pmu.c                        | 988 +++++++++++++++++++++
 10 files changed, 1404 insertions(+), 1 deletion(-)
---
base-commit: c03c1a14507dfaf8b856c2608f9ac216021fc222
change-id: 20261101-i6500-cm3-pmu-4a11f399b430

Best regards,
--  
Vladimir Kondratiev <vladimir.kondratiev@xxxxxxxxxxxx>