Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

[K Prateek Nayak]

Hello Mike,

On 4/18/2024 9:54 PM, Mike Galbraith wrote:
> Greetings,
> 
> I tossed a couple rocks at it today, and seem to have hit the little
> bugger.  The root cause seems to be doing the delay dequeue business on
> exiting tasks.  Hunk #1 of hacklet below seems to quell the explosions.
> 
> crash> bt -sx
> PID: 21722    TASK: ffff88815710ee40  CPU: 6    COMMAND: "hackbench"
>  #0 [ffff88822773bb90] machine_kexec+0x156 at ffffffff810642d6
>  #1 [ffff88822773bbe0] __crash_kexec+0xd7 at ffffffff81152a07
>  #2 [ffff88822773bc98] crash_kexec+0x23 at ffffffff81152eb3
>  #3 [ffff88822773bca8] oops_end+0xbe at ffffffff810329be
>  #4 [ffff88822773bcc8] page_fault_oops+0x81 at ffffffff81071951
>  #5 [ffff88822773bd28] exc_page_fault+0x62 at ffffffff8194e9e2
>  #6 [ffff88822773bd50] asm_exc_page_fault+0x22 at ffffffff81a00ba2
>     [exception RIP: pick_next_task_fair+178]
>     RIP: ffffffff810d8d12  RSP: ffff88822773be00  RFLAGS: 00010006
>     RAX: ffff88813cb780b8  RBX: ffff88840edb0e80  RCX: 0000000000000000
>     RDX: 0000000000000000  RSI: ffff88813cb78080  RDI: ffff88840ec30f00
>     RBP: ffff88813cb78000   R8: ffff88815710eec0   R9: 0000000000000001
>     R10: ffff88822773bdc8  R11: 0000000000000013  R12: 0000000000030e80
>     R13: ffff88815710ee40  R14: ffff88813cb78080  R15: ffff88815710ee40
>     ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
>  #7 [ffff88822773be28] __schedule+0x10d at ffffffff8195865d
>  #8 [ffff88822773be98] do_task_dead+0x3e at ffffffff810cc00e
>  #9 [ffff88822773beb0] do_exit+0x770 at ffffffff8108f0e0
> #10 [ffff88822773bf00] do_group_exit+0x2c at ffffffff8108f64c
> #11 [ffff88822773bf28] __x64_sys_exit_group+0x14 at ffffffff8108f6f4
> #12 [ffff88822773bf30] do_syscall_64+0x57 at ffffffff8194ac37
> #13 [ffff88822773bf50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b
>     RIP: 00007f4f2aa76136  RSP: 00007ffcbba84748  RFLAGS: 00000246
>     RAX: ffffffffffffffda  RBX: 0000000000000000  RCX: 00007f4f2aa76136
>     RDX: 0000000000000000  RSI: 000000000000003c  RDI: 0000000000000000
>     RBP: 00007f4f2ab86970   R8: 00000000000000e7   R9: ffffffffffffff80
>     R10: 0000000000000004  R11: 0000000000000246  R12: 00007f4f2ab86970
>     R13: 0000000000000001  R14: 00007f4f2ab8a328  R15: 0000000000000000
>     ORIG_RAX: 00000000000000e7  CS: 0033  SS: 002b
> crash> task_struct ffff88815710ee40 | grep sched_delayed
>     sched_delayed = 1 '\001',
> crash>
> 

My machine too survived the spawn test with the below patch. Thank you
for looking into this. I'll resume testing with the below patch
included.

--
Thanks and Regards,
Prateek

> ---
>  kernel/sched/fair.c |    5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
>  		update_curr(cfs_rq);
> 
>  		if (sched_feat(DELAY_DEQUEUE) && sleep &&
> +		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
>  		    !entity_eligible(cfs_rq, se)) {
>  			if (cfs_rq->next == se)
>  				cfs_rq->next = NULL;
> @@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
>  	}
> 
>  	struct sched_entity *se = pick_eevdf(cfs_rq);
> -	if (se->sched_delayed) {
> +	while (se && se->sched_delayed) {
>  		dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
>  		SCHED_WARN_ON(se->sched_delayed);
>  		SCHED_WARN_ON(se->on_rq);
>  		if (sched_feat(DELAY_ZERO) && se->vlag > 0)
>  			se->vlag = 0;
> 
> -		return NULL;
> +		se = pick_eevdf(cfs_rq);
>  	}
>  	return se;
>  }
>