Re: [PATCH v5 1/2] selftests/resctrl: Implement cl_flush() and sb() for aarch64

[Ben Horgan]

Hi Richard,

On 04/11/2026 06:43, Richard Cheng wrote:
> On Wed, Oct 21, 2026 at 10:55:35AM +0800, Ben Horgan wrote:
>> Hi Richard,
>>
>> On 20/10/2026 17:30, Ben Horgan wrote:
>>> Hi Richard,
>>>
>>> Thanks for having a look at the arm64 side of the resctrl selftests. That hasn't had much love but
>>> it would be good to have them working properly.
>>>
>>> On 12/10/2026 08:47, Richard Cheng wrote:
>>>> cl_flush() and sb() compile to empty functions on aarch64, making
>>>> mem_flush() a silent no-op and leaving CMT/CAT tests operating on
>>>> unflushed state.
>>>>
>>>> Add "dsb sy" for sb(), ARM requires a DSB whose access type covers both
>>>> loads and stores.
>>>
>>> I guess this is referring to the synchronization required for the cache maintenance instruction but
>>> that's not what you are using in this patch.
>>>
>>>>
>>>> For cl_flush(), "dc civac" only reaches the Point of Coherency, on
>>>> ARM MPAM systems the SLC lies past the PoC and may be a NOP on coherent
>>>> platforms. Instead, dirty each cacheline with a store so that
>>>> mem_flush's full-buffer sweep evicts lines from the SLC via LRU
>>>> pressure.
>>>>
>>>> Signed-off-by: Richard Cheng <icheng@xxxxxxxxxx>
>>>> ---
>>>>  tools/testing/selftests/resctrl/fill_buf.c | 11 +++++++++++
>>>>  1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/tools/testing/selftests/resctrl/fill_buf.c b/tools/testing/selftests/resctrl/fill_buf.c
>>>> index b9fa7968cd6e..eae0262fe265 100644
>>>> --- a/tools/testing/selftests/resctrl/fill_buf.c
>>>> +++ b/tools/testing/selftests/resctrl/fill_buf.c
>>>> @@ -27,6 +27,9 @@ static void sb(void)
>>>>  #if defined(__i386) || defined(__x86_64)
>>>>  	asm volatile("sfence\n\t"
>>>>  		     : : : "memory");
>>>> +#elif defined(__aarch64__)
>>>> +	asm volatile("dsb sy\n\t"
>>>> +		     : : : "memory");
>>>>  #endif
>>>>  }
>>>>  
>>>> @@ -35,6 +38,14 @@ static void cl_flush(void *p)
>>>>  #if defined(__i386) || defined(__x86_64)
>>>>  	asm volatile("clflush (%0)\n\t"
>>>>  		     : : "r"(p) : "memory");
>>>> +#elif defined(__aarch64__)
>>>> +	/*
>>>> +	 * Dirty the cache line with a store. As mem_flush() sweeps
>>>> +	 * the full test buffer (sized larger than the SLC), cache pressure
>>>> +	 * evicts lines from the SLC via LRU replacement.
>>>> +	 */
>>>> +	asm volatile("strb wzr, [%0]\n\t"
>>>> +		     : : "r"(p) : "memory");
>>>>  #endif
>>
>> One more thing that occurred to me is that if this is intended to evict all existing data in the
>> cache then it would require that the cache portions bitmap for the control group (PARTID) in use is
>> set to all 1s. E.g. 'L3:0=0xffff'. Otherwise, you may not affect the whole cache. This likely means
>> that the cl_flush() function isn't the right abstraction for MPAM and the test needs a bit of
>> restructuring to get this to fit in elegantly. Similarly, to the limbo handler in resctrl you could
>> then check the llc_occupancy() to check it's under a "low enough" threshold.
>>
>> Thanks,
>>
>> Ben
>>
> 
> Hi Ben,
> 
> Thanks for the feedback. I checked the code and ran some tests.
> You're right, my implementation writes into the benchmark's own buffer using
> the same PARRID/PMG, and the buffer isn't always larger than the SLC.
> 
> I can't claim that it produces an eviction sweep 100%.
> 
> I think what are want to do is to check whether reducing a group's cache
> allocation causes more cache misses, thus the data left in cache from the
> previous allocation must not affect the next measurement. That is also
> why I want to reuse the concept of Intel's test with CLFLUSH before
> warming the buffer under the new allocation.
> 
> I'm thinking maybe the assumption can't be achieve on ARM, I ran tests to
> measure occupancy before and after each preparation method.
> 
> I tried this patch's implement, and "DC CIVAC" since I think it can affectively
> invalidate cacheline without throwing old data away, which should left
> occupancy almost unchanged.
> And one more method with a separate large buffer in another group.
> 
> The result turns out like the following
> 
> Preparation                     Before (MiB)   After (MiB)
> DC CIVAC + DSB SY                     51.19         50.47
> zero-byte stores + DSB SY          53.12         56.43
> Separate-buffer pressure             53.25          7.14
> 
> 
> The current method increased the benchmark group's occupancy instead of
> reducing it. "DC CIVAC" left occupancy almost unchanged.
> The separate buffer method reduced occupancy very much, but still left data
> behind.
> 
> It makes me think we should reconsider the test method. I think the goal is
> to check whether the MPAM allocation limit works, clearing the cache is only
> how the Intel RDT test prepares for that check.
> 
> How about we set a group's mask before its worker initialize the buffer and

We'll still need to make sure that the PARTID used for the test hasn't got high occupancy in the
cache before running the benchmark.

> then run sustained cache traffic and measure occupancy after it settles?

I assume the occupancy is measured via resctrl, llc_occupancy.

> Maybe compare full, half and quarter masks to see whether the allocation limits take effect.
> 
> Does that sound like a useful direction for MPAM test for you ?

Yes, I think so.

Would it end up like this?

1. Choose a test control group and ensure that it doesn't have high cache occupancy using llc_occupancy
(If it does have high cache occupancy then clean it. Probably by using separate buffer with a
separate control group)
2. Configure the L3 cache bitmap for the test control group
3. Run a benchmark
4. Check that the llc_occupancy is as expected
5. Repeat 1-4 with different cache bitmaps.

However, it doesn't sound specific to MPAM but more like a general test for the cache portion
bitmap. Possibly it can be a separate cross-architecture resctrl selftest that compliments the
(somewhat) intel specific cat_test. (There isn't a platform independent way of measuring slc cache
misses on arm64.) This depends of the resctrl selftest maintainers view.

Thanks,

Ben

> 
> Best regards,
> Richard Cheng.
> 
> 
>  
>>>
>>> Isn't this cl_flush() used without changing the resctrl control and monitor group (PARTID/PMG)?
>>> Hence, I expect you are ending up making the cache full of data associated with the identifiers that
>>> the benchmark runs with. Is that the case?
>>>
>>> I'm also unclear how this the cat_test will detect any performance differences on arm/MPAM as the
>>> perf events would need to specific to the system level cache implementation.
>>>
>>> Thanks,
>>>
>>> Ben
>>>
>>>
>>>
>>>>  }
>>>>  
>>>
>>