Re: [PATCH 01/17] sched/docs: Document cpu_paravirt_mask and Paravirt CPU concept

[Shrikanth Hegde]

On 11/20/25 3:18 AM, Hillf Danton wrote:
> On Wed, 19 Nov 2025 18:14:33 +0530 Shrikanth Hegde wrote:
> > Add documentation for new cpumask called cpu_paravirt_mask. This could
> > help users in understanding what this mask and the concept behind it.
> >
> > Signed-off-by: Shrikanth Hegde <sshegde@xxxxxxxxxxxxx>
> > ---
> >   Documentation/scheduler/sched-arch.rst | 37 ++++++++++++++++++++++++++
> >   1 file changed, 37 insertions(+)
> >
> > diff --git a/Documentation/scheduler/sched-arch.rst b/Documentation/scheduler/sched-arch.rst
> > index ed07efea7d02..6972c295013d 100644
> > --- a/Documentation/scheduler/sched-arch.rst
> > +++ b/Documentation/scheduler/sched-arch.rst
> > @@ -62,6 +62,43 @@ Your cpu_idle routines need to obey the following rules:
> >   arch/x86/kernel/process.c has examples of both polling and
> >   sleeping idle functions.
> >   
> > +Paravirt CPUs
> > +=============
> > +
> > +Under virtualised environments it is possible to overcommit CPU resources.
> > +i.e sum of virtual CPU(vCPU) of all VM's is greater than number of physical
> > +CPUs(pCPU). Under such conditions when all or many VM's have high utilization,
> > +hypervisor won't be able to satisfy the CPU requirement and has to context
> > +switch within or across VM. i.e hypervisor need to preempt one vCPU to run
> > +another. This is called vCPU preemption. This is more expensive compared to
> > +task context switch within a vCPU.
> > +
> What is missing is
> 1) vCPU preemption is X% more expensive compared to task context switch within a vCPU.

This would change from arch to arch IMO. Will try to get numbers from PowerVM hypervisor.

> > +In such cases it is better that VM's co-ordinate among themselves and ask for
> > +less CPU by not using some of the vCPUs. Such vCPUs where workload can be
> > +avoided at the moment for less vCPU preemption are called as "Paravirt CPUs".
> > +Note that when the pCPU contention goes away, these vCPUs can be used again
> > +by the workload.
> > +
> 2) given X, how to work out Y, the number of Paravirt CPUs for the simple
> scenario like 8 pCPUs and 16 vCPUs (8 vCPUs from VM1, 8 vCPUs from VM2)?

Y need not be dependent on X. Note CPUs are marked as paravirt only when both VM's
end up consuming all the CPU resource.

Different cases:
1. VM1 is idle and VM2 is idle - No vCPUs are marked as paravirt.
2. VM1 is 100% busy and VM2 is idle - No steal time seen - No vCPUs is marked as paravirt.
3. VM1 is idle and VM2 is 100% busy - No steal time seen - No vCPUs is marked as paravirt.
4. VM1 is 100% busy and VM2 is 100% busy - 50% steal time would be seen in each -
	Since there are only 8 pCPUs (assuming each VM1 is allocated equally), 4 vCPUs in
	each VM will be marked as paravirt. Workload consolidates to remaining 4 vCPUs and
	hence no steal time will seen. Benefit would seen since host doesn't need to change
	expensive VM context switches.

> > +Arch need to set/unset the specific vCPU in cpu_paravirt_mask. When set, avoid
> > +that vCPU and when unset, use it as usual.
> > +
> > +Scheduler will try to avoid paravirt vCPUs as much as it can.
> > +This is achieved by
> > +1. Not selecting paravirt CPU at wakeup.
> > +2. Push the task away from paravirt CPU at tick.
> > +3. Not selecting paravirt CPU at load balance.
> > +
> > +This works only for SCHED_RT and SCHED_NORMAL. SCHED_EXT and userspace can make
> > +choices accordingly using cpu_paravirt_mask.
> > +
> > +/sys/devices/system/cpu/paravirt prints the current cpu_paravirt_mask in
> > +cpulist format.
> > +
> > +Notes:
> > +1. A task pinned only on paravirt CPUs will continue to run there.
> > +2. This feature is available under CONFIG_PARAVIRT
> > +3. Refer to PowerPC for architecure implementation side.
> > +4. Doesn't push out any task running on isolated CPUs.
> >   
> >   Possible arch/ problems
> >   =======================
> > --
> > 2.47.3