Re: [RFC PATCH 0/6] mm/damon: hardware-sampled access reports

[KunWu Chan]

On Sat, Aug 22, 2026 at 12:32 AM Ravi Jonnalagadda
<ravis.opensrc@xxxxxxxxx> wrote:
>
> On Fri, Aug 21, 2026 at 2:55 AM KunWu Chan <kunwu.chan@xxxxxxxxx> wrote:
> >
> > Hi Ravi,
> >
> > Thanks for the detailed reply.  The fan-in diagnosis makes perfect sense.
> >
> > > Your series is based on the v1 substrate, and the way an event is
> > > configured is the part that moves most.
> > >
> > > [...]
> >
> > Glad to hear v2 is on the way.  The probe-based model is actually
> > something I encountered while building the AUX backend. When multiple
> > sources can feed the same region (e.g. PTE scanning + SPE/IBS), the
> > question of how to combine their reports becomes interesting.
> >
> > For example, CXL tiering may care about physical-address locality, while
> > per-process monitoring may need virtual-address attribution.  How do you
> > see probe weights being determined — is that expected to be
> > scenario-specific, or is there a more general principle behind the
> > weighting?
>
> Hello KunWu,
>
> SJ can answer that authoritatively -- the probe and weight interface
> is his. Here is my understanding which SJ can confirm later:
>

Hi Ravi,

Thanks for your detailed reply and for sharing your understanding.
The probe-weight framing clears up the part I had backwards.

> Weights are relative, and only among probes. A context can hold up to four;
> each report carries its probe index and credits that probe's own hit count,
> so two PMU-based probes in one context are scored in proportion to their
> weights, and that is where a scenario-specific choice belongs.
>

This makes sense. I had initially been thinking about using the weights to
tune SPE against PTE scanning within the same context. I now understand that
the weights are for combining multiple PMU probes within a context, rather
than blending PMU reports with the existing PTE-based access checking.

> What does not mix is a weighted probe with the other two sources. Any
> nonzero weight switches the whole context to probe-weighted scoring, which
> turns page-table access checking off -- so SPE cannot be blended with PTE
> scanning, it replaces it. Page-fault reports carry no probe index; they
> credit the region's access rate but have no hit count, so under weighted
> scoring they are not part of the score either. Both coexist with a probe
> only at weight zero, where everything feeds the same access rate and is
> indistinguishable afterwards.
>

The "replace, not blend" distinction was the key correction for me.
For PA locality versus VA attribution, using separate contexts therefore
makes more sense.

> So for your case I would read it as: PA locality and VA attribution are
> different targets and different address kinds, which points at separate
> contexts rather than one context with tuned weights.
>
> >
> > > 2. The 67.9% drop rate is fan-in, not sizing
> > >
> > > ---
> > >
> > > [...]
> >
> > I completely missed this.  The observability framework showed the
> > symptom (67.9% drop) but got the root cause wrong — it's not a ring
> > sizing issue, it's a fan-in problem.  With a single kdamond thread
> > draining the per-CPU sources, the samples effectively get funneled
> > into a single ring, which explains the numbers.
> >
> > The per-CPU drain approach using `system_percpu_wq` and `queue_work_on()`
> > looks like the right fix.  Our AUX backend already carries a CPU argument
> > through the ops, so adapting it should be straightforward.  I'll take a
> > look at your v2 branch and re-test with the per-CPU drain.
> >
> > Two questions:
> > 1. Is `damon/perf-rfc-v2-08-20-26` ready for me to rebase the
> >    AUX backend and observability framework on top of it?  I'd like to
> >    validate the per-CPU drain fix with real SPE hardware.
> >
>
> Yes:
>
>   https://github.com/ravis-opensrc/linux.git
>   branch damon/perf-rfc-v2-08-20-26, tip 9c0410564596
>
> That is the branch I used to exercise the AUX path. It carries all five of
> your AUX kernel-consumer API patches unmodified, and from the SPE series,
> patch 1/4 plus the generic half of 2/4. 2/4's SPE backend and parser, and
> 3/4 and 4/4 which test them, are SPE-specific and I used an x86 box
> with virtual PMU.
>

Thanks for making the testing branch available. I'll use it as the base
for the rebase and testing below.

> The branch is force-updated post fixing two trailers please re-fetch
> if necessary.
>
> None of it is meant for my v2 posting.
> This branch is only the vehicle for testing the AUX report path.
>
> The SHAs you will care about:
>
>   808519650a7a  the per-CPU report side
>   e3323a3bdc75  your AUX interface, ported to the probe model
>   1c347519c626  parse each CPU's AUX buffer on that CPU
>   9c0410564596  perf/core: balance the AUX buffer reference on kernel
>                 release
>

Thanks for this detailed information.

> 1c347519c626 drains with one work item per CPU in the event's cpumask, via
> queue_work_on() on system_percpu_wq -- system_wq would not do, being
> unbound it silently ignores the CPU targeting.
>

Understood. This also matches the fan-in issue you pointed out earlier.
I'll keep the CPU-local drain model when reworking the AUX backend.

> Two things came out of a quick test in a VM:
>
> - spe_backend_init() calls perf_event_setup_aux(), but I could not find any
>   path calling perf_event_release_aux() -- spe_backend_cleanup() frees only
>   st->win, and your kerneldoc says the release must happen before
>   perf_event_release_kernel(). Left alone on my branch.
>

Thanks for catching this. I'll trace the cleanup path and make sure the
missing `perf_event_release_aux()` is called at the appropriate point before
`perf_event_release_kernel()`, then re-test the lifetime path.

> - 9c0410564596: the explicit ring_buffer_put() in perf_event_release_aux()
>   is one too many, since ring_buffer_attach(event, NULL) just above already
>   drops the event's reference.
>

Thanks for catching this as well. I'll keep `9c0410564596` when
rebasing the AUX work.

> > 2. For the probe-based configuration, does the `prep_action = "perf_event"`
> >    model preserve the same event lifecycle?  Our AUX path needs to create
> >    and bind the event during apply, keep it disabled initially, and enable
> >    it when kdamond starts.
> >
>
> Yes -- created disabled during apply, bound, then enabled. Per CPU, on the
> kdamond turn-on path:
>
>   perf_event_create_kernel_counter()   attr has .pinned = 1, .disabled = 1,
>                                        so the counter is installed but not
>                                        yet schedulable
>   ops->init(event, cpu, perf_event)    your backend attaches the AUX buffer
>                                        here, via perf_event_setup_aux()
>   perf_event_enable()                  only now can the PMU be scheduled in
>
> So the buffer is always attached before the counter starts. If ops->init()
> fails the counter is released without ever being enabled.
> kernel/watchdog_perf.c uses the same pinned + disabled-then-enable pairing.
>

This lifecycle is exactly what the AUX path needs: the AUX buffer is
attached during `ops->init()` while the event is still disabled, and
the event is only enabled after successful initialization.

> Sanity-tested both report paths in a VM.
>
> Best Regards,
> Ravi.
>

I'm looking forward to the formal v2 as well. I'll continue working on
the perf-side changes so that the new model can be exercised with
hardware-sampled access reports in practice.

My next steps are:

1. Re-fetch `damon/perf-rfc-v2-08-20-26` and rebase my AUX backend and
   observability framework onto the probe model, including the SPE-specific
   backend/parser and tests.

2. Integrate the per-CPU AUX drain from `1c347519c626`, using
   `queue_work_on()` on `system_percpu_wq`, and validate that the fan-in
   problem is gone.

3. Fix the AUX lifetime path by making sure `perf_event_release_aux()`
   is called at the appropriate point before `perf_event_release_kernel()`.
   I'll also keep `9c0410564596` for the `ring_buffer_put()` fix.

4. Re-test the complete lifecycle and the per-CPU drain on real ARM SPE,
   including stop/restart, CPU hotplug, and failure paths, and compare the
   observability counters with the previous 67.9% drop result.

I'll report back with the results.

Thanks,
KunWu