Re: [PATCH v5 18/29] sched/deadline: Move a task's bandwidth only where room was made

From: Waiman Long

Date: Fri Nov 13 2026 - 13:39:42 EST


On 11/13/26 8:51 AM, Mathieu Desnoyers wrote:
Hi Waiman,

On 2026-11-12 23:37, Waiman Long wrote:
For cpuset v2, it is not possible for a cpuset to span two root domains.
I believe it is not possible for v1 too, though I am not 100% sure.
Please let me know if you are aware of a way to do that.
It happens with isolcpus="domain". The CPUs it isolates stay in the
effective CPUs of the top cpuset, and of any cpuset that is not a
partition root. They are in the default root domain. The other CPUs
are in another root domain.

The message should have named isolcpus. The planned next version does.
Right, top_cpuset is the exception here. Boot isolated CPUs are not allowed in a root partition. They are only allowed in isolated partitions. So root partition other than the top one cannot span more than one root domain.

This is on tip sched/urgent (v7.4-rc1 based), in a 5-CPU guest booted
with isolcpus=domain,1,2. A is a plain cgroup v2 cpuset:

# cat /sys/devices/system/cpu/isolated
1-2
# echo 0-4 > A/cpuset.cpus
# cat A/cpuset.cpus.effective
0-4
# cat A/cpuset.cpus.partition
member

Two tasks in A call sched_setattr(SCHED_DEADLINE). Task 95 is pinned to
CPUs 0,3,4 and asks for 10%. Task 98 is pinned to CPUs 1,2 and asks for
20%. Both are admitted. /sys/kernel/debug/sched/debug gives total_bw of
the root domain of each CPU. "Before" is the fair servers alone:

CPU before with the two tasks
0,3,4 157284 262141 (+10%)
1,2 104856 314571 (+20%)

So the two tasks are in the same cpuset and are counted in two root
domains. Then task 95 is given CPU 1 alone:

# taskset -p 2 95
taskset: can't set pid 95's affinity: Device or resource busy
# echo -1 > /proc/sys/kernel/sched_rt_runtime_us
# taskset -p 2 95
pid 95's new affinity mask: 2

After that, total_bw is 157284 for CPUs 0,3,4 and still 314571 for
CPUs 1,2. The 10% of task 95 is counted nowhere. That is the case the
patch fixes.

I did not test cgroup v1.

The planned next version has a new cpuset patch for the same
configuration. A deadline task in such a cpuset can also change root
domain when its cpuset loses CPUs. The patch refuses that:
- a write to cpuset.cpus fails with EBUSY if it leaves a deadline task
no CPU in the root domain that counts it;
- cpuset makes a partition root invalid if it would do the same.

We already have pending cpuset patch that will disallow taking the last housekeeping CPU from the top_cpuset partition.  The partition will be disabled in that patch if users attempt to do so. So it is effectively changing the root domain to that of the parent cpuset.

As said above, a partition below the top_cpuset cannot span more than one root domain. Are you referring to the case that shrinking the cpuset CPU count to reduce the total DL bandwidth will fail with EBUSY when it is less than what the existing DL tasks require?

Thanks,
Longman

My reply to Juri on patch 20 describes it with two related patches:
https://lore.kernel.org/all/e21aaf40-b929-444c-b707-3200e21d7301@xxxxxxxxxxxx/

I would like your opinion on these two refusals before I post the
patch.

Thanks,

Mathieu