RE: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR metrics

[Besar Wicaksono]

> -----Original Message-----
> From: Ian Rogers <irogers@xxxxxxxxxx>
> Sent: Tuesday, October 13, 2026 9:30 PM
> To: Besar Wicaksono <bwicaksono@xxxxxxxxxx>
> Cc: Chun-Tse Shao <ctshao@xxxxxxxxxx>; acme@xxxxxxxxxx;
> namhyung@xxxxxxxxxx; peterz@xxxxxxxxxxxxx; mingo@xxxxxxxxxx;
> mark.rutland@xxxxxxx; alexander.shishkin@xxxxxxxxxxxxxxx;
> jolsa@xxxxxxxxxx; adrian.hunter@xxxxxxxxx; james.clark@xxxxxxxxxx; linux-
> perf-users@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; Matt Ochs
> <mochs@xxxxxxxxxx>; Sean Kelley <skelley@xxxxxxxxxx>; Rich Wiley
> <rwiley@xxxxxxxxxx>; Vikram Sethi <vsethi@xxxxxxxxxx>; Raj Joshi
> <rajjoshi@xxxxxxxxxx>
> Subject: Re: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR
> metrics
> 
> External email: Use caution opening links or attachments
> 
> 
> On Tue, Oct 13, 2026 at 2:07 PM Besar Wicaksono
> <bwicaksono@xxxxxxxxxx> wrote:
> >
> > Hi Chun-Tse,
> >
> > I tested this patch on 2-socket Tegra410. Both DDR bandwidth and
> > read latency metrics give correct values in the default, --per-socket,
> > and -A modes.
> >
> > Some small comments inline below.
> >
> > > -----Original Message-----
> > > From: Chun-Tse Shao <ctshao@xxxxxxxxxx>
> > > Sent: Monday, September 28, 2026 5:51 PM
> > > To: acme@xxxxxxxxxx; namhyung@xxxxxxxxxx; irogers@xxxxxxxxxx
> > > Cc: peterz@xxxxxxxxxxxxx; mingo@xxxxxxxxxx;
> mark.rutland@xxxxxxx;
> > > alexander.shishkin@xxxxxxxxxxxxxxx; jolsa@xxxxxxxxxx;
> > > adrian.hunter@xxxxxxxxx; james.clark@xxxxxxxxxx; Besar Wicaksono
> > > <bwicaksono@xxxxxxxxxx>; linux-perf-users@xxxxxxxxxxxxxxx; linux-
> > > kernel@xxxxxxxxxxxxxxx; Chun-Tse Shao <ctshao@xxxxxxxxxx>
> > > Subject: [PATCH 1/2] perf jevents: Add NVIDIA Tegra410 uncore DDR
> > > metrics
> > >
> > > External email: Use caution opening links or attachments
> > >
> > >
> > > Add DDR bandwidth and latency metrics for the NVIDIA Tegra410 SoC.
> The
> > > formulas follow Documentation/admin-guide/perf/nvidia-tegra410-
> > > pmu.rst:
> > >
> > >   AVG_MEM_READ_BANDWIDTH_IN_GBPS = MEM_BYTES_RD /
> > > ELAPSED_TIME_IN_NS
> > >   AVG_MEM_WRITE_BANDWIDTH_IN_GBPS = MEM_BYTES_WR /
> > > ELAPSED_TIME_IN_NS
> > >
> > >   FREQ_IN_GHZ = CYCLES / ELAPSED_TIME_IN_NS
> > >   AVG_LATENCY_IN_CYCLES = RD_CUM_OUTS / RD_REQ
> > >   AVERAGE_LATENCY_IN_NS = AVG_LATENCY_IN_CYCLES / FREQ_IN_GHZ
> > >
> > > The bandwidth metrics use the mem_bytes_rd/mem_bytes_wr events
> of
> > > the
> > > Unified Coherence Fabric (UCF) PMU with source/destination filters:
> > >
> > >   lpm_ddr_bw, lpm_ddr_rd_bw, lpm_ddr_wr_bw: all sources to local
> CMEM.
> > >   lpm_ddr_loc_rd_bw, lpm_ddr_loc_wr_bw: local CPU and non-CPU
> sources
> > > to
> > >     local CMEM.
> > >   lpm_ddr_rem_rd_bw, lpm_ddr_rem_wr_bw: local CPU and non-CPU
> > > sources to
> > >     remote memory.
> > >
> > > The latency metrics use the CPU Memory (CMEM) latency PMU:
> > >
> > >   lpm_ddr_lat, lpm_ddr_rd_lat: average DDR read latency in ns.
> > >   lpm_ddr_wr_lat: the PMU only measures reads, so this is always 0.
> > >
> >
> > A question on naming: what does the "lpm_" prefix stand for? Is it the
> > convention for all metrics generated by the Python scripts (I see it on
> > the Intel/AMD and common metrics too)?
> 
> Hi Besar, naming is one of the hardest problems in Computer Science
> :-) When we first added the metrics there was a request to
> differentiate between metrics coming from the python scripts and
> metrics coming from vendor code, like Intel top-down. We agreed on
> having a prefix, and lpm_ was suggested as meaning Linux perf metric.
> All of those words seem redundant given the metric is being measured
> on Linux using perf, but it was adopted as the best suggestion. If you
> have a better proposal, I think we can change it.
> 

Hi Ian,

Thanks for the explanation, this makes sense, and it keeps them distinguishable
from the vendor JSON metrics.

Regards,
Besar

> Thanks,
> Ian
> 
> > > There is one CMEM latency PMU per socket and perf sums the counts of
> the
> > > PMUs in an aggregation, e.g. of both sockets in the default aggregation
> > > mode. Use aggr_nr() to divide the summed cycles by the number of
> PMUs,
> > > otherwise the frequency is too high and the latency too low by that
> > > factor.
> > >
> > > The events are sysfs events of the nvidia_ucf_pmu_<socket-id> and
> > > nvidia_cmem_latency_pmu_<socket-id> PMUs rather than json events,
> so
> > > add
> > > the "nvidia" PMU prefix to the prefixes that skip the named json event
> > > check in metric.py.
> > >
> > > pmu-events/Build only runs arm64_metrics.py for the "arm" vendor
> > > models,
> > > so also run it for the "nvidia" vendor models to generate the Tegra410
> > > metrics.
> > >
> > > Signed-off-by: Chun-Tse Shao <ctshao@xxxxxxxxxx>
> > > Assisted-by: Gemini:gemini-3.1-pro-preview
> > > ---
> > >  tools/perf/pmu-events/Build            |  2 +-
> > >  tools/perf/pmu-events/arm64_metrics.py | 66
> > > +++++++++++++++++++++++++-
> > >  tools/perf/pmu-events/metric.py        |  2 +-
> > >  3 files changed, 66 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
> > > index 1ac067be9d34..8dc8cfcb8b04 100644
> > > --- a/tools/perf/pmu-events/Build
> > > +++ b/tools/perf/pmu-events/Build
> > > @@ -78,7 +78,7 @@ endif
> > >
> > >  ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all))
> > >  # Generate ARM Json
> > > -ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn)
> > > +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/* pmu-
> > > events/arch/arm64/nvidia/*|grep -v cmn)
> > >  ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-
> metrics.json)
> > >  ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-
> > > metricgroups.json)
> > >  GEN_JSON += $(ARM_METRICS) $(ARM_METRICGROUPS)
> > > diff --git a/tools/perf/pmu-events/arm64_metrics.py b/tools/perf/pmu-
> > > events/arm64_metrics.py
> > > index 4ecda96d11fa..b0d0651fec12 100755
> > > --- a/tools/perf/pmu-events/arm64_metrics.py
> > > +++ b/tools/perf/pmu-events/arm64_metrics.py
> > > @@ -2,14 +2,75 @@
> > >  # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> > >  import argparse
> > >  import os
> > > -from metric import (JsonEncodeMetric,
> > > JsonEncodeMetricGroupDescriptions, LoadEvents,
> > > -                    MetricGroup)
> > > +from typing import Optional
> > > +from metric import (aggr_nr, d_ratio, Event, JsonEncodeMetric,
> > > +                    JsonEncodeMetricGroupDescriptions, LoadEvents, Metric,
> > > MetricGroup)
> > >  from common_metrics import Cycles
> > >
> > >  # Global command line arguments.
> > >  _args = None
> > >
> > >
> > > +def NvidiaT410Uncore() -> Optional[MetricGroup]:
> > > +    """Uncore metrics for the NVIDIA Tegra410 SoC.
> > > +
> > > +    The formulas follow Documentation/admin-guide/perf/nvidia-
> tegra410-
> > > pmu.rst.
> > > +    """
> > > +    assert _args is not None
> > > +    if _args.vendor != "nvidia" or _args.model != "t410":
> > > +        return None
> > > +
> > > +    interval_sec = Event("duration_time")
> > > +    mb_per_sec = f"{1/1e6}MB/s"
> > > +
> > > +    def DdrBw() -> MetricGroup:
> > > +        # Unified Coherence Fabric (UCF) PMU:
> > > +        # AVG_MEM_{READ,WRITE}_BANDWIDTH = MEM_BYTES_{RD,WR}
> /
> > > ELAPSED_TIME
> > > +        loc_src = "src_loc_cpu=0x1,src_loc_noncpu=0x1"
> > > +        rd = Event("nvidia_ucf_pmu/mem_bytes_rd,dst_loc_cmem=0x1/")
> > > +        wr =
> Event("nvidia_ucf_pmu/mem_bytes_wr,dst_loc_cmem=0x1/")
> > > +        loc_rd =
> > > Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_loc_cmem=0x1/")
> > > +        loc_wr =
> > >
> Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_loc_cmem=0x1/")
> > > +        rem_rd =
> > > Event(f"nvidia_ucf_pmu/mem_bytes_rd,{loc_src},dst_rem=0x1/")
> > > +        rem_wr =
> > > Event(f"nvidia_ucf_pmu/mem_bytes_wr,{loc_src},dst_rem=0x1/")
> > > +        return MetricGroup("lpm_ddr_bw", [
> > > +            Metric("lpm_ddr_bw", "Total DDR bandwidth",
> > > +                   d_ratio(rd + wr, interval_sec), mb_per_sec),
> > > +            Metric("lpm_ddr_rd_bw", "DDR read bandwidth",
> > > +                   d_ratio(rd, interval_sec), mb_per_sec),
> > > +            Metric("lpm_ddr_wr_bw", "DDR write bandwidth",
> > > +                   d_ratio(wr, interval_sec), mb_per_sec),
> > > +            Metric("lpm_ddr_loc_rd_bw", "Local DDR read bandwidth",
> > > +                   d_ratio(loc_rd, interval_sec), mb_per_sec),
> > > +            Metric("lpm_ddr_loc_wr_bw", "Local DDR write bandwidth",
> > > +                   d_ratio(loc_wr, interval_sec), mb_per_sec),
> > > +            Metric("lpm_ddr_rem_rd_bw", "Remote DDR read bandwidth",
> > > +                   d_ratio(rem_rd, interval_sec), mb_per_sec),
> > > +            Metric("lpm_ddr_rem_wr_bw", "Remote DDR write bandwidth",
> > > +                   d_ratio(rem_wr, interval_sec), mb_per_sec),
> > > +        ], description="NVIDIA Tegra410 DDR bandwidth")
> > > +
> > > +    def DdrLat() -> MetricGroup:
> > > +        # CPU Memory (CMEM) latency PMU, one per socket:
> > > +        # AVERAGE_LATENCY_IN_NS = (RD_CUM_OUTS / RD_REQ) /
> (CYCLES /
> > > ELAPSED_TIME_IN_NS)
> > > +        rd_cum_outs = Event("nvidia_cmem_latency_pmu/rd_cum_outs/")
> > > +        rd_req = Event("nvidia_cmem_latency_pmu/rd_req/")
> > > +        cycles = Event("nvidia_cmem_latency_pmu/cycles/")
> > > +        # perf sums the counts of the PMUs in an aggregation, e.g. both
> > > sockets
> > > +        # by default, so divide cycles by their number to get the frequency.
> > > +        rd_lat = d_ratio(1e9 * interval_sec * rd_cum_outs * aggr_nr(cycles),
> > > rd_req * cycles)
> > > +        return MetricGroup("lpm_ddr_lat", [
> > > +            Metric("lpm_ddr_lat", "Average DDR read latency in ns", rd_lat,
> > > "1ns"),
> > > +            Metric("lpm_ddr_rd_lat", "DDR read latency in ns", rd_lat, "1ns"),
> >
> > Could we keep just lpm_ddr_rd_lat ?
> >
> > > +            # The CMEM latency PMU only measures reads, write latency is
> > > always 0.
> > > +            Metric("lpm_ddr_wr_lat", "DDR write latency in ns",
> > > +                   rd_cum_outs - rd_cum_outs, "1ns"),
> > > +        ], description="NVIDIA Tegra410 DDR latency")
> >
> > This always prints 0.0 ns, which reads like a real measurement of
> > zero-latency writes. Since the hardware can't measure write latency, I'd
> > drop this metric rather than report a placeholder.
> >
> > Tested-by: Besar Wicaksono <bwicaksono@xxxxxxxxxx>
> >
> > Thanks,
> > Besar
> >
> > > +
> > > +    return MetricGroup("lpm_uncore", [DdrBw(), DdrLat()],
> > > +                       description="NVIDIA Tegra410 uncore metrics")
> > > +
> > > +
> > >  def main() -> None:
> > >      global _args
> > >
> > > @@ -37,6 +98,7 @@ def main() -> None:
> > >
> > >      all_metrics = MetricGroup("", [
> > >          Cycles(),
> > > +        NvidiaT410Uncore(),
> > >      ])
> > >
> > >      if _args.metricgroups:
> > > diff --git a/tools/perf/pmu-events/metric.py b/tools/perf/pmu-
> > > events/metric.py
> > > index f5d81eccd5c3..6fb69024d1fd 100644
> > > --- a/tools/perf/pmu-events/metric.py
> > > +++ b/tools/perf/pmu-events/metric.py
> > > @@ -91,7 +91,7 @@ def CheckEveryEvent(*names: str) -> None:
> > >        name = name[:name.find(':')]
> > >      elif '/' in name:
> > >        name = name[:name.find('/')]
> > > -      if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power',
> > > 'cha', 'uncore']]):
> > > +      if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power',
> > > 'cha', 'uncore', 'nvidia']]):
> > >          continue
> > >      if name not in all_events_all_models:
> > >        raise ValueError(f"Is {name} a named json event?")
> > > --
> > > 2.56.0.rc1.315.gc6ed9934b7-goog
> >