Re: [PATCH v5 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu

[James Morse]

Hi Reinette,

On 09/08/2023 23:38, Reinette Chatre wrote:
> On 7/28/2023 9:42 AM, James Morse wrote:
>> When a CPU is taken offline resctrl may need to move the overflow or
>> limbo handlers to run on a different CPU.
>>
>> Once the offline callbacks have been split, cqm_setup_limbo_handler()
>> will be called while the CPU that is going offline is still present
>> in the cpu_mask.
>>
>> Pass the CPU to exclude to cqm_setup_limbo_handler() and
>> mbm_setup_overflow_handler(). These functions can use a variant of
>> cpumask_any_but() when selecting the CPU. -1 is used to indicate no CPUs
>> need excluding.
>>
>> A subsequent patch moves these calls to be before CPUs have been removed,
>> so this exclude_cpus behaviour is temporary.

>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>> index c0b1ad8d8f6d..471cdc4e4eae 100644
>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c

>> @@ -816,15 +817,28 @@ void cqm_handle_limbo(struct work_struct *work)
>>  	mutex_unlock(&rdtgroup_mutex);
>>  }
>>  
>> -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms)
>> +/**
>> + * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this
>> + *                             domain.
>> + * @delay_ms:      How far in the future the handler should run.
>> + * @exclude_cpu:   Which CPU the handler should not run on,
>> + *		   RESCTRL_PICK_ANY_CPU to pick any CPU.
>> + */
>> +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms,
>> +			     int exclude_cpu)
>>  {
>>  	unsigned long delay = msecs_to_jiffies(delay_ms);
>>  	int cpu;
>>  
>> -	cpu = cpumask_any_housekeeping(&dom->cpu_mask);
>> +	if (exclude_cpu == RESCTRL_PICK_ANY_CPU)
>> +		cpu = cpumask_any_housekeeping(&dom->cpu_mask);
>> +	else
>> +		cpu = cpumask_any_housekeeping_but(&dom->cpu_mask,
>> +						   exclude_cpu);
> 
> Having callers need to do this checking seems unnecessary and makes the
> code complicated. Can cpumask_any_housekeeping_but() instead be made
> slightly smarter to handle the case where exclude_cpu == RESCTRL_PICK_ANY_CPU ?
> 
> Looks like there is a bit of duplication between 
> cpumask_any_housekeeping() and cpumask_any_housekeeping_but().

Yup, this was because I was originally going to add them to cpumask.h, but figured it
would be easier to leave them here - in a shape that could be moved to cpumask.h if anyone
else needs them.

Using one helper for both would simplify things for resctrl, I'll do that.


Thanks,

James