Re: [PATCH v3 17/38] x86/resctrl: Stop using the for_each_*_rdt_resource() walkers

[James Morse]

Hi Reinette,

On 01/07/2024 22:10, Reinette Chatre wrote:
> On 7/1/24 11:16 AM, James Morse wrote:
>> On 28/06/2024 17:48, Reinette Chatre wrote:
>>> On 6/14/24 8:00 AM, James Morse wrote:
>>>> The for_each_*_rdt_resource() helpers walk the architecture's array
>>>> of structures, using the resctrl visible part as an iterator. These
>>>> became over-complex when the structures were split into a
>>>> filesystem and architecture-specific struct. This approach avoided
>>>> the need to touch every call site.
>>>>
>>>> Once the filesystem parts of resctrl are moved to /fs/, both the
>>>> architecture's resource array, and the definition of those structures
>>>> is no longer accessible. To support resctrl, each architecture would
>>>> have to provide equally complex macros.
>>>>
>>>> Change the resctrl code that uses these to walk through the resource_level
>>>> enum and check the mon/alloc capable flags instead. Instances in core.c,
>>>> and resctrl_arch_reset_resources() remain part of x86's architecture
>>>> specific code.
>>
>>>> diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
>>>> b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
>>>> index aacf236dfe3b..ad20822bb64e 100644
>>>> --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
>>>> +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
>>>> @@ -854,7 +855,11 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
>>>>         * First determine which cpus have pseudo-locked regions
>>>>         * associated with them.
>>>>         */
>>>> -    for_each_alloc_capable_rdt_resource(r) {
>>>> +    for (i = 0; i < RDT_NUM_RESOURCES; i++) {
>>>> +        r = resctrl_arch_get_resource(i);
>>>> +        if (!r->alloc_capable)
>>>> +            continue;
>>>> +
>>>
>>> This looks like enough duplicate boilerplate for a new macro. For simplicity the
>>> macro could require two arguments with enum resctrl_res_level also provided?
>>
>> I was hoping to escape from these clever macros! If you think this is too much:
>> - we'd need to come up with another name, as the arch code keeps the existing definition.
>> - to avoid touching every caller, it needs doing without an explicit iterator variable.
>>
>> I guess the cleanest thing is to redefine the existing macros to use
>> resctrl_arch_get_resource(). Putting this in include/linxu/resctrl.h at least avoids each
>> architecture needing to define these, or forcing it to use an array.
>>
>> The result is slightly more readable than the current version:
>> | #define for_each_rdt_resource(_r)                              \
>> |        for (_r = resctrl_arch_get_resource(0);                 \
>> |             _r->rid < RDT_NUM_RESOURCES;                       \
>> |             _r = resctrl_arch_get_resource(_r->rid + 1))
>>
>> This leans heavily on resctrl_arch_get_resource() not being able to return NULL, and
>> having to return a dummy resource that is neither alloc nor mon capable. We may need to
>> revisit that if it becomes a burden for the arch code.
> 
> Replacing the repetitive four lines of code with a single line seems good to me.

> resctrl_arch_get_resource() being able to return NULL is introduced in this series but
> I am not seeing any handling of a possible NULL value. Not being able to return NULL thus
> already seems a requirement?

It's currently implicit because until this point resctrl has just reached into the
rdt_resources_all[] array - and can never get a NULL pointer. Replacing that with a helper
needed to preserve the no-NULLs behaviour.
Changing this created too much churn so the resctrl idiom is to check
alloc_enabled/mon_enabled to see if the resource actually exists....

If we wanted to change this, that for_each_rdt_resource() would need an index variable as
_r could be NULL.


Thanks,

James