Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

[Shaopeng Tan (Fujitsu)]

Hello Reinette,


>>>> Second, regarding the naming of "mbm_total_bytes",
>>>> the meaning of this name seems to differ from what is typically found under mon_data/mon_L3_<id>/mbm_total_bytes.
>>>
>>> Could you please elaborate how the meaning is different? The only information I have about
>>> the systems needing this is in Nvidia's portion of
>>> https://lpc.events/event/19/contributions/2093/attachments/1958/4172/resctrl%20Microconference%20LPC%202025%20Tokyo.pdf
>>>
>>>> To avoid confusion and better reflect its different nature,
>>>> how about considering an alternative name such as mbm_global_bytes or another more appropriate identifier.
>>
>> Current MBM events in resctrl are per L3 to monitor MB between L3 and memory,
>> and mbm_total_bytes monitors the L3 total external bandwidth to the next level of the memory hierarchy.
>> Is my understanding of this correct?
>
>It is my understanding also.
>
>>
>> Based on NVIDIA's information, they even assume that CPU-less nodes will not feature an L3 cache.
>> I believe NVIDIA's objective is to collect traffic data for each NODE/NUMA,
>> without needing to distinguish which specific L3 cache the traffic originated from.
>
>That is what associating monitoring with scope (instead of L3 resource), and supporting node scope, aims
>to address.
>
>>
>> This aligns with the fact that, in ARM architectures (e.g., on NVIDIA's platforms or Fujitsu's MONAKA),
>> the MPAM MSC (Memory System Component) can also be located on the memory controller.
>> In such cases,it becomes impossible to distinguish which L3 cache the traffic came from.
>
>This is not clear to me. Is the goal to expose the bandwidth at each memory controller?

The goal is to expose the bandwidth of each NUMA/NODE.
In this context, my intention was to explain why it is impossible to distinguish which L3 cache the traffic came from.

>>
>> Therefore, the proposed 'mbm_global_bytes' refers to account for traffic from all L3 cache within the global system.
>>
>
>As I see it the directory name would make the monitoring scope clear, whether it is
>at L3 scope or NUMA node scope. Within the directory there would be the individual files
>that should be interpreted based on the scope of the directory they are in. I thus
>see a "mbm_total_bytes" in a "L3 scope" directory to have different meaning
>from a "mbm_total_bytes" in a "NUMA node scope" directory.

I understand. 
Your suggestion is to clarify the monitoring scope (L3 or NUMA node) by the directory name, 
and that individual files within that directory would be interpreted based on its scope. 
If the scope is "L3", then "mbm_total_bytes" would mean collecting traffic data for each L3, 
regardless of which NUMA node the traffic is destined for.
If the scope is "NUMA node", then "mbm_total_bytes" would mean collecting traffic data for each node/NUMA, 
regardless of which L3 cache the traffic originates from.
I believe this approach will be clear as long as it's clearly documented.  Thanks.

Also, regarding the scope for NUMA nodes, 
since it's not possible to distinguish which L3 cache the traffic came from,
it also appears that support for `event_configs` is not necessary.
(There is no need to configure "non-local NUMA" or "local NUMA".)


Best regards,
Shaopeng TAN