[BUG] cpuset: WARNING in cpuset_update_sd_hk_unlock() when isolated partitions take every housekeeping CPU

[Mathieu Desnoyers]

Hi,

With isolcpus=domain, cpuset accepts isolated partitions that leave no
housekeeping CPU. housekeeping_update() then refuses the new mask, and
cpuset_update_sd_hk_unlock() warns.

I found this while testing a fix for another cpuset problem [1]. Its
sequence goes through that problem first, so I ran it only on a kernel
with that fix.

Trigger
=======

Boot 5 CPUs with isolcpus=domain,0,1. CPUs 2 to 4 are then the
housekeeping CPUs. With cgroup v2:

  cd /sys/fs/cgroup
  echo +cpuset > cgroup.subtree_control
  mkdir P Q
  echo 4 > P/cpuset.cpus
  echo isolated > P/cpuset.cpus.partition
  echo 2-3 > Q/cpuset.cpus
  echo root > Q/cpuset.cpus.partition
  echo isolated > Q/cpuset.cpus.partition    # WARNING

The last write is accepted. Q reads back as isolated.

  WARNING: kernel/cgroup/cpuset.c:1481 at cpuset_update_sd_hk_unlock+0x79/0x80, CPU#1: k2/93
  CPU: 1 UID: 0 PID: 93 Comm: k2 Not tainted 7.4.0-rc1-00268-ge99d0e004b05 #108 PREEMPT(lazy)
  RAX: 00000000ffffffea
  Call Trace:
   cpuset_partition_write+0x9e/0xe0
   kernfs_fop_write_iter+0x14d/0x210
   vfs_write+0x252/0x550
   ksys_write+0x6c/0xe0
   do_syscall_64+0x109/0x5d0

RAX is -EINVAL, the return of housekeeping_update().

Analysis
========

>From reading tip sched/urgent (006991aa8997):
- Making Q isolated would remove CPUs 2 and 3 from HK_TYPE_DOMAIN.
  CPUs 0 and 1 are isolated at boot, and P holds CPU 4. So no
  housekeeping CPU would remain.
- housekeeping_update() refuses that mask with -EINVAL. Commit
  03ff73510169 ("cpuset: Update HK_TYPE_DOMAIN cpumask from cpuset")
  calls it under WARN_ON_ONCE() in cpuset_update_sd_hk_unlock().
- prstate_housekeeping_conflict() refuses such a partition only with
  nohz_full, since commit 103b08709e8a ("cgroup/cpuset: Fail if
  isolated and nohz_full don't leave any housekeeping").

So cpuset and the housekeeping code disagree on what is valid. The
write returns 0, and the isolated CPUs of the top cpuset read 0-4. But
by reading, HK_TYPE_DOMAIN keeps its old mask.

Should cpuset refuse an isolated partition that leaves no
HK_TYPE_DOMAIN housekeeping CPU, as it does for nohz_full? I did not
write a patch, since you may prefer another place for the check.

The analysis was written by an LLM coding assistant under my direction
and review.

Thanks,

Mathieu

[1] The empty sched domain report, sent along with this one.