Re: [PATCH v4 00/41] arm_mpam: Add KVM/arm64 and resctrl glue code
From: Ben Horgan
Date: Tue Feb 24 2026 - 12:53:30 EST
Hi Zeng,
On 2/24/26 14:19, Ben Horgan wrote:
> Hi Zeng,
>
> On 2/24/26 11:03, Zeng Heng wrote:
>> Hi Ben,
>>
>> On 2026/2/16 20:22, Ben Horgan wrote:
>>> Hi Zeng,
>>>
>>> On 2/14/26 09:40, Zeng Heng wrote:
>>>> Hi Ben,
>>>>
>>>> On 2026/2/4 5:43, Ben Horgan wrote:
>>> [...]
>>>>>
>>>>> Based on:
>>>>> [1] git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/cache
>>>>> (To include telemetry code which changes the resctrl arch interface)
>>>>>
>>>>> The series can be retrieved from:
>>>>> https://gitlab.arm.com/linux-arm/linux-bh.git mpam_resctrl_glue_v4
>>>>> (Final commit is a fix already in 6.19-rc8)
>>>>>
>>> [...]
>>>>>
>>>>
>>>> I've tested the MPAM functionality on my local Kunpeng platform. Here's
>>>> a summary of the results:
>>>
>>> Thank you very much for your testing and detailed report.
>>>
>>>>
>>>> Features enabled and verified:
>>>> * L2 and L3 CPBM
>>>> * L3 CSU
>>>> * L2 and L3 CDP
>>>> All enabled features passed functional testing as expected.
>>>>
>>>> + Tested-by: Zeng Heng <zengheng4@xxxxxxxxxx>
>>>>
>>>> Features not enabled:
>>>> 1. MATA MBMAX partition and MBWU monitor.
>>>
>>> What's MATA here? Just memory allocation or something more specific?
>>>
>>
>> The MATA serves as the agent for the Broadway MESI coherence protocol
>> among multiple L3 caches or between I/O and L3 caches. It maintains data
>> coherence among multiple L3 caches or between I/O and L3 caches.
>>
>> As the connection module between the SoC and the memory system, the MATA
>> interfaces with the DMC on the DDR die. It provides the system with DDR
>> access paths, delivering high-bandwidth, low-latency DDR read/write
>> access.
>>
>> On the Kunpeng chip, the MB related MSC resides in this module rather
>> than being directly located on the L3 cache controller.
>>
>>>> Reason: These do not meet the driver's current topology>
>>> expectations for MB support, hence they were not initialized.
>>>> This behavior is expected.
>>>
>>> Is this because you have more than 1 L3 cache?
>>
>> Yes, Kunpeng platform has more than 1 L3 cache.
>>
>> However, the reason it is not supported here is that the current driver
>> does not support MATA recognition, while both Kunpeng MBM and MBA
>> functionalities reside in the MATA MSC side as mentioned above,
>> resulting in the inability to provide support.
>>
>> Relevant logs are as follows:
>>
>> [ 10.997406] mpam:topology_matches_l3: class 255 component 0 has
>> Mismatched CPU mask with L3 equivalent
>> [ 10.997411] mpam:mpam_resctrl_pick_mba: class 255 topology doesn't
>> match L3
>>
>>>
>>>>
>>>> 2. L2 CSU and MBWU monitors.
>>>> Reason: The current MPAM driver does not support L2-related
>>>> functionality yet.
>>>
>>> Thanks for letting us know you have these. But, yes, unfortunately
>>> monitoring is only supported on the L3 at the moment.
>>>
>>>>
>>>> + Tested-by: Zeng Heng <zengheng4@xxxxxxxxxx>
>>>>
>>>>
>>>> Detailed test logs are as follows:
>>>
>>> I'm confused by these logs as it looks like you aren't getting any
>>> monitors but you verified the L3 CSU. Also, it looks like cpor (cbpm) is
>>> disabled (at least partially) but you verified L2 and L3 CPBM. Is this
>>> across different machines?
>>>
>>
>> The logs were of course tested on the same machine.
>
> Ok and the same run? Just asking because I can't yet see how this all
> goes together.
>
>>
>> Since the L3 CSU/CPBM resides on the L3 cache, it can be correctly
>> recognized and run smoothly.
>>
>> In fact, not only was L2 successfully mounted, but all L2 MSC CPBMs were
>> also correctly recognized. The suspicion here is that the L2
>> class->components were incorrectly mounted to an unknown object, which
>> is believed to be related to the monitoring capabilities (CSU and MBWU)
>> of Kunpeng L2. The root cause is still being investigated.
>
> I'll try and mock up a system with L2 monitoring and cpbm to see if the
> driver behaves in the same way.
>
> However, I still can't understand how you get CPOR on L2 after seeing
> "mpam:mpam_resctrl_pick_caches: class 2 cache misses CPOR" as that
> should be the only place class is set for the l2 cache which guards the
> call to mpam_control_init() in mpam_resctrl_setup(). The
> mpam_monitor_init() comes later so I can't see how that effects.
>
> Is the driver corrupting something or writing to mpam_resctrl_controls
> with the wrong index? Clutching at straws here.
>
>>
>> Resctrl mounting status:
>>
>> # cat schemata
>> L2:4=ff;7=ff;10=ff;13=ff;16=ff;19=ff;22=ff;25=ff;29=ff;32=ff;35=ff;
>> 38=ff;41=ff;44=ff;47=ff;50=ff;54=ff;57=ff;60=ff;63=ff;66=ff;69=ff;72=ff;
>> 75=ff;79=ff;82=ff;85=ff;88=ff;91=ff;94=ff;97=ff;100=ff;104=ff;107=ff;
>> 110=ff;113=ff;116=ff;119=ff;122=ff;125=ff;129=ff;132=ff;135=ff;138=ff;
>> 141=ff;144=ff;147=ff;150=ff;154=ff;157=ff;160=ff;163=ff;166=ff;169=ff;
>> 172=ff;175=ff;179=ff;182=ff;185=ff;188=ff;191=ff;194=ff;197=ff;200=ff;
>> 204=ff;207=ff;210=ff;213=ff;216=ff;219=ff;222=ff;225=ff;229=ff;232=ff;
>> 235=ff;238=ff;241=ff;244=ff;247=ff;250=ff;254=ff;257=ff;260=ff;263=ff;
>> 266=ff;269=ff;272=ff;275=ff;279=ff;282=ff;285=ff;288=ff;291=ff;294=ff;
>> 297=ff;300=ff
>> L3:1=1ffff;26=1ffff;51=1ffff;76=1ffff;101=1ffff;126=1ffff;151=1ffff;
>> 176=1ffff;201=1ffff;226=1ffff;251=1ffff;276=1ffff
>>
>> # ls mon_data/*/*
>> mon_data/mon_L3_01/llc_occupancy mon_data/mon_L3_151/llc_occupancy
>> mon_data/mon_L3_226/llc_occupancy mon_data/mon_L3_276/llc_occupancy
>> mon_data/mon_L3_101/llc_occupancy mon_data/mon_L3_176/llc_occupancy
>> mon_data/mon_L3_251/llc_occupancy mon_data/mon_L3_51/llc_occupancy
>> mon_data/mon_L3_126/llc_occupancy mon_data/mon_L3_201/llc_occupancy
>> mon_data/mon_L3_26/llc_occupancy mon_data/mon_L3_76/llc_occupancy
>
> Thanks for the extra details. These are as expected, right?
>
>>
>>>>
>>>> Boot logs:
>>>> [root@localhost ~]# dmesg | grep -i mpam
>>>> [ 0.000000] ACPI: MPAM 0x000000007FF35018 003024 (v01 HISI HIP12
>>>> 00000000 HISI 20151124)
>>>> [ 9.509852] mpam_msc mpam_msc.64: Merging features for
>>>> vmsc:0xffff0800973cf5a0 |= ris:0xffff08009757ee90
>>>> [ 9.509859] mpam_msc mpam_msc.254: Merging features for
>>>> class:0xffff08009736fe50 &= vmsc:0xffff080097628520
>>>> [ 9.509860] mpam:__props_mismatch:
>>>> mpam_has_feature(mpam_feat_cpor_part, parent) = 1
>>>> [ 9.509864] mpam:__props_mismatch:
>>>> mpam_has_feature(mpam_feat_cpor_part, child) = 0
>>>> [ 9.509866] mpam:__props_mismatch: parent->cpbm_wd = 8
>>>> [ 9.509869] mpam:__props_mismatch: child->cpbm_wd = 0
>>>> [ 9.509871] mpam:__props_mismatch: alias = 0
>>>> [ 9.509873] mpam:__props_mismatch: cleared cpor_part
>
> Do you know where this mismatch is happening? Is it expected?
I have added some of James' debug patches at:
https://gitlab.arm.com/linux-arm/linux-bh/-/tree/mpam_resctrl_glue_v5_debugfs?ref_type=heads
These are on top of a v5 of the series. They apply to older series
as well.
The debugfs should then contain details of all the MSC which
should help debugging.
It would be great if you were able to share that information, either on list or
privately if needed.
You can snapshot the mpam debugfs using:
find /sys/kernel/debug/mpam -type f -exec sh -c 'echo {}; cat {}' \; > mpam_debugfs.txt
>
>>>
>>> cpor (partially) disabled?
>>>
>>>> [ 9.509876] mpam:__props_mismatch: took the min num_csu_mon
>>>> [ 9.509878] mpam:__props_mismatch: took the min num_mbwu_mon
>>>> [ 9.509881] mpam_msc mpam_msc.252: Merging features for
>>
>> [...]
>>
>>>> [ 10.978496] mpam:mpam_resctrl_pick_caches: class 2 cache misses CPOR
>>>
>>> No L2 cpor?
>
> This particularly confuses me...
>
>>>
>>>> [ 10.978497] mpam:mpam_resctrl_pick_caches: class 255 is not a cache
>>>> [ 10.980470] mpam:mpam_resctrl_pick_mba: class 2 is before L3
>>>> [ 10.980472] mpam:mpam_resctrl_pick_mba: class 3 has no bandwidth
>>>> control
>>>> [ 10.997406] mpam:topology_matches_l3: class 255 component 0 has
>>>> Mismatched CPU mask with L3 equivalent
>>>> [ 10.997411] mpam:mpam_resctrl_pick_mba: class 255 topology doesn't
>>>> match L3
>>>> [ 10.997415] mpam:mpam_resctrl_pick_counters: class 2 is before L3
>>>> [ 11.024109] mpam:topology_matches_l3: class 3 component 276 has
>>>> Mismatched CPU mask with L3 equivalent
>>>> [ 11.024114] mpam:class_has_usable_mbwu: monitors usable in free-
>>>> running mode
>>>
>>> mbwu enabled?
>>
>> The fact that the number of monitors merely satisfies the conditions for
>> free-running mode does not imply that the MBWU functionality can be
>> successfully mounted. The specific reasons are explained above.
>
> True
>
>>
>>>
>>>> [ 11.063882] mpam:topology_matches_l3: class 255 component 0 has
>>>> Mismatched CPU mask with L3 equivalent
>>>> [ 11.113183] mpam:mpam_resctrl_alloc_domain: Skipped monitor domain
>>>> online - no monitors
>>>> [ 11.113189] MPAM enabled with 32 PARTIDs and 4 PMGs
>>>>
>>>>
>>
>> Sorry for the late reply. And this is my first day back from a long
>> vacation.
>
> No problem. Hope you had a good holiday.
>
>>
>>
>> Best regards,
>> Zeng Heng
>
> Thanks,
>
> Ben
>
>
Thanks,
Ben