RE: sched/fair: Long delays starting RT processes on idle cpu.

[David Laight]

From: linux-kernel-owner@xxxxxxxxxxxxxxx <linux-kernel-owner@xxxxxxxxxxxxxxx> On Behalf Of Steven Rostedt
> Sent: 27 January 2020 17:16
> On Mon, 27 Jan 2020 16:56:12 +0000
> David Laight <David.Laight@xxxxxxxxxx> wrote:
> 
> > >From Steven Rostedt
> > > Sent: 27 January 2020 15:50
> > > On Mon, 27 Jan 2020 15:39:24 +0000
> > > David Laight <David.Laight@xxxxxxxxxx> wrote:
> > >
> > > > I'd have thought that the processor should wake up much faster than that.
> > > > I can't see the memory write that is paired with the monitor/mwait.
> > > > Does it need a strong barrier?
> > >
> > > You may want to prevent the CPU from going into a deep C state. 90us is
> > > something I would expect if the CPU is in a deep C state (I've seen
> > > much longer wake up times due to deep C state).
> > >
> > > Boot the kernel with idle=poll and see if you can trigger the same
> > > latency. If not, then you know it's the CPU going into a deep C state
> > > that is causing your latency.
> >
> > With idle=poll the delays seem to be minimal.
> >
> > Is there any way to limit the C state entered by mwait?
> 
> I believe you can dynamically change C state. There's a pdf about it:
> 
>   https://wiki.ntb.ch/infoportal/_media/embedded_systems/ethercat/controlling_processor_c-state_usage_in_linux_v1.1_nov2013.pdf

Writing 20 to /dev/cpu_dma_latency (and holding the fd open) removes the excessive latency.
The latency values (for my system) seem to be 0, 1, 10, 59, 80.
I suspect the '10' - C1E (halt + energy saving) is not as bad as expected.

While it may well be that the cpu doesn't enter the C3/C6/C7 states if we
are doing a reasonable amount of work, they are a PITA when you are
trying to look at how a system behaves.

While searching for 'cpu_dma_latency' in Documentation does give some info,
it isn't entirely obvious that is what you need to look for.
Even finding the intel_idle.c code doesn't help much.
A reference from 'man setpriority'  might be a hint....

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)