Re: [PATCH v2 1/2] selftests/resctrl: Adjust effective L3 cache size with SNC enabled
From: Reinette Chatre
Date: Fri May 31 2024 - 12:18:16 EST
Hi Tony and Maciej,
On 5/30/24 5:34 PM, Luck, Tony wrote:
When SNC mode is enabled the effective amount of L3 cache available
for allocation is divided by the number of nodes per L3.
This was a mistake in original implementation and no longer done.
My original kernel code adjusted value reported in the "size" file in resctrl.
That's no longer done because the effective size depends on how applications
are allocating and using memory. Since the kernel can't know that, it
seemed best to just report the total size of the cache.
But I think the resctrl tests still need to take this into account when running
llc_occupancy tests.
E.g. on a 2-way SNC system with a 100MB L3 cache a test that allocates
memory from its local SNC node (default behavior without using libnuma)
will only see 50 MB llc_occupancy with a fully populated L3 mask in the
schemata file.
This seems to contradict the "Cache and memory bandwidth allocation features
continue to operate at the scope of the L3 cache." statement from [1]?
I'll clean that up. MBA isn't affected. But cache allocation is affected in that
the amount of cache represented by each bit in the masks in the schemata
file is reduced by a factor equal to SNC nodes per L3 cache.
Thanks Tony. I trust that this is what Maciej intended since the change
is specifically named "Adjuct _effective_ L3 cache size". I'd like to
recommend that your comments be added before the change to
get_cache_size() ...
/*
* The amount of cache represented by each bit in the masks
* in the schemata file is reduced by a factor equal to SNC
* nodes per L3 cache.
* E.g. on a SNC-2 system with a 100MB L3 cache a test that
* allocates memory from its local SNC node (default behavior
* without using libnuma) will only see 50 MB llc_occupancy
* with a fully populated L3 mask in the schemata file.
*/
Reinette