Re: [PATCH v3] sched/core: Fix uclamp ABI bug, clean up and robustify sched_read_attr() ABI logic and code

[Thadeu Lima de Souza Cascardo]

On Wed, Sep 04, 2019 at 12:39:25PM +0200, Ingo Molnar wrote:
> 
> * Dietmar Eggemann <dietmar.eggemann@xxxxxxx> wrote:
> 
> > > -v3 attached. Build and minimally boot tested.
> > > 
> > > Thanks,
> > > 
> > > 	Ingo
> > > 
> > 
> > This patch fixes the issue (almost).
> > 
> > LTP's sched_getattr01 passes again. But IMHO the prio 'chrt -p $$'
> > should be 0 instead of -65536.
> 
> Yeah, I forgot to zero-initialize the structure ...
> 
> Does the attached -v4 work any better for you?
> 
> Thanks,
> 
> 	Ingo

Hi, Ingo.

Thanks for the patch. It works just fine for me, I have only one comment about
it, below.

Acked-by: Thadeu Lima de Souza Cascardo <cascardo@xxxxxxxxxxxxx>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@xxxxxxxxxxxxx>

I also tested LTP, chrt, and that 5.2 (before the breaking commit) and 5.3 with
your patch behave the same when the user size is larger than what the kernel
knows, the return is 0 and the struct size is set to the known value.

There is one odd behaviour now, which is whenever size is set between VER0 and
VER1, we will have a partial struct filled up, instead of getting E2BIG or
EINVAL.

> 
> ===============================>
> Date: Wed, 4 Sep 2019 09:55:32 +0200
> Subject: [PATCH] sched/core: Fix uclamp ABI bug, clean up and robustify sched_read_attr() ABI logic and code
> 
> Thadeu Lima de Souza Cascardo reported that 'chrt' broke on recent kernels:
> 
>   $ chrt -p $$
>   chrt: failed to get pid 26306's policy: Argument list too long
> 
> and he has root-caused the bug to the following commit increasing sched_attr
> size and breaking sched_read_attr() into returning -EFBIG:
> 
>   a509a7cd7974 ("sched/uclamp: Extend sched_setattr() to support utilization clamping")
> 
> The other, bigger bug is that the whole sched_getattr() and sched_read_attr()
> logic of checking non-zero bits in new ABI components is arguably broken,
> and pretty much any extension of the ABI will spuriously break the ABI.
> That's way too fragile.
> 
> Instead implement the perf syscall's extensible ABI instead, which we
> already implement on the sched_setattr() side:
> 
>  - if user-attributes have the same size as kernel attributes then the
>    logic is unchanged.
> 
>  - if user-attributes are larger than the kernel knows about then simply
>    skip the extra bits, but set attr->size to the (smaller) kernel size
>    so that tooling can (in principle) handle older kernel as well.
> 
>  - if user-attributes are smaller than the kernel knows about then just
>    copy whatever user-space can accept.
> 
> Also clean up the whole logic:
> 
>  - Simplify the code flow - there's no need for 'ret' for example.
> 
>  - Standardize on 'kattr/uattr' and 'ksize/usize' naming to make sure we
>    always know which side we are dealing with.
> 
>  - Why is it called 'read' when what it does is to copy to user? This
>    code is so far away from VFS read() semantics that the naming is
>    actively confusing. Name it sched_attr_copy_to_user() instead, which
>    mirrors other copy_to_user() functionality.
> 
>  - Move the attr->size assignment from the head of sched_getattr() to the
>    sched_attr_copy_to_user() function. Nothing else within the kernel
>    should care about the size of the structure.
> 
> With these fixes the sched_getattr() syscall now nicely supports an
> extensible ABI in both a forward and backward compatible fashion, and
> will also fix the chrt bug.
> 
> As an added bonus the bogus -EFBIG return is removed as well, which as
> Thadeu noted should have been -E2BIG to begin with.
> 
> Reported-by: Thadeu Lima de Souza Cascardo <cascardo@xxxxxxxxxxxxx>
> Cc: Arnaldo Carvalho de Melo <acme@xxxxxxxxxxxxx>
> Cc: Jiri Olsa <jolsa@xxxxxxxxxx>
> Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
> Cc: Patrick Bellasi <patrick.bellasi@xxxxxxx>
> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Fixes: a509a7cd7974 ("sched/uclamp: Extend sched_setattr() to support utilization clamping")
> Link: https://lkml.kernel.org/r/20190904075532.GA26751@xxxxxxxxx
> Signed-off-by: Ingo Molnar <mingo@xxxxxxxxxx>
> ---
>  kernel/sched/core.c | 82 +++++++++++++++++++++++++++--------------------------
>  1 file changed, 42 insertions(+), 40 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 010d578118d6..3c64eb28dd70 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -5105,39 +5105,44 @@ SYSCALL_DEFINE2(sched_getparam, pid_t, pid, struct sched_param __user *, param)
>  	return retval;
>  }
>  
> -static int sched_read_attr(struct sched_attr __user *uattr,
> -			   struct sched_attr *attr,
> -			   unsigned int usize)
> +/*
> + * Copy the kernel size attribute structure (which might be larger
> + * than what user-space knows about) to user-space.
> + *
> + * Note that all cases are valid: user-space buffer can be larger or
> + * smaller than the kernel-space buffer. The usual case is that both
> + * have the same size.
> + */
> +static int
> +sched_attr_copy_to_user(struct sched_attr __user *uattr,
> +			struct sched_attr *kattr,
> +			unsigned int usize)
>  {
> -	int ret;
> +	unsigned int ksize = sizeof(*kattr);
>  
> -	if (!access_ok(uattr, usize))
> +	if (!access_ok(uattr, ksize))
>  		return -EFAULT;
>  

I believe this should be either usize or kattr->size and moved below (or just
reuse min(usize,ksize)).

Otherwise, an old binary that uses the old ABI (that is, VER0 as size) may get
EFAULT here. This should almost never in practice. I tried, but I could hardly
use an address that would fail access_ok. But, theoretically, that still breaks
ABI.

Regards.
Cascardo.

>  	/*
> -	 * If we're handed a smaller struct than we know of,
> -	 * ensure all the unknown bits are 0 - i.e. old
> -	 * user-space does not get uncomplete information.
> +	 * sched_getattr() ABI forwards and backwards compatibility:
> +	 *
> +	 * If usize == ksize then we just copy everything to user-space and all is good.
> +	 *
> +	 * If usize < ksize then we only copy as much as user-space has space for,
> +	 * this keeps ABI compatibility as well. We skip the rest.
> +	 *
> +	 * If usize > ksize then user-space is using a newer version of the ABI,
> +	 * which part the kernel doesn't know about. Just ignore it - tooling can
> +	 * detect the kernel's knowledge of attributes from the attr->size value
> +	 * which is set to ksize in this case.
>  	 */
> -	if (usize < sizeof(*attr)) {
> -		unsigned char *addr;
> -		unsigned char *end;
> -
> -		addr = (void *)attr + usize;
> -		end  = (void *)attr + sizeof(*attr);
> -
> -		for (; addr < end; addr++) {
> -			if (*addr)
> -				return -EFBIG;
> -		}
> +	kattr->size = min(usize, ksize);
>  
> -		attr->size = usize;
> -	}
> -
> -	ret = copy_to_user(uattr, attr, attr->size);
> -	if (ret)
> +	if (copy_to_user(uattr, kattr, kattr->size))
>  		return -EFAULT;
>  
> +	printk("sched_attr_copy_to_user(): copied %d bytes. (ksize: %d, usize: %d)\n", kattr->size, ksize, usize);
> +
>  	return 0;
>  }
>  
> @@ -5145,20 +5150,18 @@ static int sched_read_attr(struct sched_attr __user *uattr,
>   * sys_sched_getattr - similar to sched_getparam, but with sched_attr
>   * @pid: the pid in question.
>   * @uattr: structure containing the extended parameters.
> - * @size: sizeof(attr) for fwd/bwd comp.
> + * @usize: sizeof(attr) that user-space knows about, for forwards and backwards compatibility.
>   * @flags: for future extension.
>   */
>  SYSCALL_DEFINE4(sched_getattr, pid_t, pid, struct sched_attr __user *, uattr,
> -		unsigned int, size, unsigned int, flags)
> +		unsigned int, usize, unsigned int, flags)
>  {
> -	struct sched_attr attr = {
> -		.size = sizeof(struct sched_attr),
> -	};
> +	struct sched_attr kattr = { };
>  	struct task_struct *p;
>  	int retval;
>  
> -	if (!uattr || pid < 0 || size > PAGE_SIZE ||
> -	    size < SCHED_ATTR_SIZE_VER0 || flags)
> +	if (!uattr || pid < 0 || usize > PAGE_SIZE ||
> +	    usize < SCHED_ATTR_SIZE_VER0 || flags)
>  		return -EINVAL;
>  
>  	rcu_read_lock();
> @@ -5171,25 +5174,24 @@ SYSCALL_DEFINE4(sched_getattr, pid_t, pid, struct sched_attr __user *, uattr,
>  	if (retval)
>  		goto out_unlock;
>  
> -	attr.sched_policy = p->policy;
> +	kattr.sched_policy = p->policy;
>  	if (p->sched_reset_on_fork)
> -		attr.sched_flags |= SCHED_FLAG_RESET_ON_FORK;
> +		kattr.sched_flags |= SCHED_FLAG_RESET_ON_FORK;
>  	if (task_has_dl_policy(p))
> -		__getparam_dl(p, &attr);
> +		__getparam_dl(p, &kattr);
>  	else if (task_has_rt_policy(p))
> -		attr.sched_priority = p->rt_priority;
> +		kattr.sched_priority = p->rt_priority;
>  	else
> -		attr.sched_nice = task_nice(p);
> +		kattr.sched_nice = task_nice(p);
>  
>  #ifdef CONFIG_UCLAMP_TASK
> -	attr.sched_util_min = p->uclamp_req[UCLAMP_MIN].value;
> -	attr.sched_util_max = p->uclamp_req[UCLAMP_MAX].value;
> +	kattr.sched_util_min = p->uclamp_req[UCLAMP_MIN].value;
> +	kattr.sched_util_max = p->uclamp_req[UCLAMP_MAX].value;
>  #endif
>  
>  	rcu_read_unlock();
>  
> -	retval = sched_read_attr(uattr, &attr, size);
> -	return retval;
> +	return sched_attr_copy_to_user(uattr, &kattr, usize);
>  
>  out_unlock:
>  	rcu_read_unlock();