Re: [PATCH v14 10/12] cpuset: Add documentation about the new "cpuset.sched.partition" flag

From: Waiman Long
Date: Tue Nov 06 2018 - 09:09:24 EST


On 11/06/2018 06:50 AM, Peter Zijlstra wrote:
> On Mon, Oct 15, 2018 at 04:29:35PM -0400, Waiman Long wrote:
>> The cgroup-v2.rst file is updated to document the purpose of the new
>> "cpuset.sched.partition" flag and how its usage.
>>
>> Signed-off-by: Waiman Long <longman@xxxxxxxxxx>
>> ---
>> Documentation/admin-guide/cgroup-v2.rst | 66 +++++++++++++++++++++++++
>> 1 file changed, 66 insertions(+)
>>
>> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
>> index 533e85cb851b..178cda473a26 100644
>> --- a/Documentation/admin-guide/cgroup-v2.rst
>> +++ b/Documentation/admin-guide/cgroup-v2.rst
>> @@ -1686,6 +1686,72 @@ Cpuset Interface Files
>>
>> Its value will be affected by memory nodes hotplug events.
>>
>> + cpuset.sched.partition
>> + A read-write single value file which exists on non-root
>> + cpuset-enabled cgroups. It accepts either "0" (off) or "1"
>> + (on) when written to.
>> + This flag is set and owned by the
>> + parent cgroup.
> What does that mean? The parent cgroup doesn't 'set' anything at all.
> The user will.
>
>> +
>> + If set, it indicates that the current cgroup is the root of a
>> + new partition or scheduling domain that comprises itself and
>> + all its descendants except those that are separate partition
>> + roots themselves and their descendants. The root cgroup is
>> + always a partition root.
>> +
>> + There are constraints on where this flag can be set. It can
>> + only be set in a cgroup if all the following conditions are true.
>> +
>> + 1) The "cpuset.cpus" is not empty and the list of CPUs are
>> + exclusive, i.e. they are not shared by any of its siblings.
>> + 2) The parent cgroup is a partition root.
>> + 3) The "cpuset.cpus" is also a proper subset of the parent's
>> + "cpuset.cpus.effective".
>> + 4) There is no child cgroups with cpuset enabled. This is for
>> + eliminating corner cases that have to be handled if such a
>> + condition is allowed.
>> +
>> + Setting this flag will take the CPUs away from the effective
>> + CPUs of the parent cgroup. Once it is set, this flag cannot
>> + be cleared if there are any child cgroups with cpuset enabled.
>> +
>> + A parent partition cannot distribute all its CPUs to its
>> + child partitions. There must be at least one cpu left in the
>> + parent partition.
>> +
>> + Once becoming a partition root, changes to "cpuset.cpus" is
>> + generally allowed as long as the first condition above is true,
>> + the change will not take away all the CPUs from the parent
>> + partition and the new "cpuset.cpus" value is a superset of its
>> + children's "cpuset.cpus" values.
>> + Sometimes, external factors like changes to ancestors'
>> + "cpuset.cpus" or cpu hotplug can cause the state of the partition
>> + root to change. On read, the "cpuset.sched.partition" file
>> + can show the following values.
> Are those the only conditions under which that -1 can happen? Parent
> taking away CPUs it previously granted and hotplug?
>
>> +
>> + "0" Not a partition root
>> + "1" Partition root
>> + "-1" Erroneous partition root
>> +
>> + It is a partition root if the first 2 partition root conditions
>> + above are true and at least one CPU from "cpuset.cpus" is
>> + granted by the parent cgroup.
>> +
>> + A partition root can become an erroneous partition root if none
>> + of CPUs requested in "cpuset.cpus" can be granted by the parent
>> + cgroup or the parent cgroup is no longer a partition root.
>> + In this case, it is not a real partition even though the
>> + restriction of the first partition root condition above will
>> + still apply. All the tasks in the cgroup will be migrated to
>> + the nearest ancestor partition.
> Effectively or actual? Actual migrating tasks out of the cgroup is
> irreversible.

I should have reworded it to emphasize that this flag is not delegatable
to the child cpuset.

>> + An erroneous partition root can be transitioned back to a real
>> + partition root if at least one of the requested CPUs can now be
>> + granted by its parent. In this case, the tasks will be migrated
>> + back to the newly created partition. Clearing the partition
>> + flag of an erroneous partition root is always allowed even if
>> + child cpusets are present.
> So you need to clarify the above point (I think it is effectively),
> because otherwise you don't know which tasks to put back.

OK, will do that.

Cheers,
Longman