Re: [PATCH v11 7/7] sched: Split scheduler and execution contexts

[Peter Zijlstra]

On Tue, Jul 09, 2024 at 01:31:50PM -0700, John Stultz wrote:
> From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> 
> Let's define the scheduling context as all the scheduler state
> in task_struct for the task selected to run, and the execution
> context as all state required to actually run the task.
> 
> Currently both are intertwined in task_struct. We want to
> logically split these such that we can use the scheduling
> context of the task selected to be scheduled, but use the
> execution context of a different task to actually be run.
> 
> To this purpose, introduce rq_selected() macro to point to the
> task_struct selected from the runqueue by the scheduler, and
> will be used for scheduler state, and preserve rq->curr to
> indicate the execution context of the task that will actually be
> run.

> * Swapped proxy for selected for clarity

I'm not loving this naming...  what does selected even mean? What was
wrong with proxy? -- (did we have this conversation before?)

> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index 493de4cc320a..7ee8c7fa0ae8 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -1051,7 +1051,7 @@ struct rq {
>  	 */
>  	unsigned int		nr_uninterruptible;
>  
> -	struct task_struct __rcu	*curr;
> +	struct task_struct __rcu	*curr;       /* Execution context */
>  	struct task_struct	*idle;
>  	struct task_struct	*stop;
>  	unsigned long		next_balance;
> @@ -1246,6 +1246,13 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
>  #define cpu_curr(cpu)		(cpu_rq(cpu)->curr)
>  #define raw_rq()		raw_cpu_ptr(&runqueues)
>  
> +/* For now, rq_selected == rq->curr */
> +#define rq_selected(rq)		((rq)->curr)
> +static inline void rq_set_selected(struct rq *rq, struct task_struct *t)
> +{
> +	/* Do nothing */
> +}
> +
>  struct sched_group;
>  #ifdef CONFIG_SCHED_CORE
>  static inline struct cpumask *sched_group_span(struct sched_group *sg);
> @@ -2151,11 +2158,25 @@ static inline u64 global_rt_runtime(void)
>  	return (u64)sysctl_sched_rt_runtime * NSEC_PER_USEC;
>  }
>  
> +/*
> + * Is p the current execution context?
> + */
>  static inline int task_current(struct rq *rq, struct task_struct *p)
>  {
>  	return rq->curr == p;
>  }
>  
> +/*
> + * Is p the current scheduling context?
> + *
> + * Note that it might be the current execution context at the same time if
> + * rq->curr == rq_selected() == p.
> + */
> +static inline int task_current_selected(struct rq *rq, struct task_struct *p)
> +{
> +	return rq_selected(rq) == p;
> +}


And I think I hated on the macros before, and you said you needed them
to to allow !PROXY builds.

But what about something like:

#ifdef CONFIG_PROXY_EXEC
	struct task_struct __rcu *proxy;
	struct task_struct __rcu *curr;
#else
	union {
		struct task_struct __rcu *proxy;
		struct task_struct __rcu *curr;
	};
#endif


And then we can use rq->proxy and rq->curr like the good old days?


I realize this is going to be a lot of typing to fix up, and perhaps
there's a reason to not do this. But...