Re: [PATCH RT] fix IPI balancing for 4.14-rt

[Sebastian Andrzej Siewior]

On 2017-11-21 10:24:36 [-0500], Steven Rostedt wrote:
> On Tue, 21 Nov 2017 09:14:25 -0600
> Clark Williams <williams@xxxxxxxxxx> wrote:
> 
> > I was testing 4.14-rt1 on a large system (cores == 96) and saw that
> > we were getting into an rt balancing storm, so I tried applying Steven's
> > patch (not upstream yet):
> > 
> >     sched/rt: Simplify the IPI rt balancing logic
> > 
> Why is this patch necessary?
> 
> Is it because you have the irq_work running in non hard irq context? I
> think you need something like this instead (if you haven't already
> added it):

I cherry-picked commit 4bdced5c9a29 ("sched/rt: Simplify the IPI based
RT balancing logic") and while refreshing the queue I noticed that the
irq_work struct moved and added the fix below into the original patch
where the IRQ_WORK_HARD_IRQ flag was added.

> -- Steve
> 
> Index: linux-rt.git/kernel/sched/topology.c
> ===================================================================
> --- linux-rt.git.orig/kernel/sched/topology.c
> +++ linux-rt.git/kernel/sched/topology.c
> @@ -257,6 +257,7 @@ static int init_rootdomain(struct root_d
>  	rd->rto_cpu = -1;
>  	raw_spin_lock_init(&rd->rto_lock);
>  	init_irq_work(&rd->rto_push_work, rto_push_irq_work_func);
> +	rd->rto_push_work.flags |= IRQ_WORK_HARD_IRQ;
>  #endif
>  
>  	init_dl_bw(&rd->dl_bw);

Sebastian