[PATCH-tip v2] sched: Use kfree_rcu() in do_set_cpus_allowed()

From: Waiman Long
Date: Mon Nov 28 2022 - 20:28:28 EST


Commit 851a723e45d1 ("sched: Always clear user_cpus_ptr in
do_set_cpus_allowed()") may call kfree() if user_cpus_ptr was previously
set. Unfortunately, some of the callers of do_set_cpus_allowed()
may have pi_lock held when calling it. So the following splats may be
printed especially when running with a PREEMPT_RT kernel:

WARNING: possible circular locking dependency detected
BUG: sleeping function called from invalid context

To avoid these problems, kfree_rcu() is used instead. An internal
cpumask_rcuhead union is created for the sole purpose of facilitating
the use of kfree_rcu() to free the cpumask.

Fixes: 851a723e45d1 ("sched: Always clear user_cpus_ptr in do_set_cpus_allowed()")
Suggested-by: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Signed-off-by: Waiman Long <longman@xxxxxxxxxx>
---
kernel/sched/core.c | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index f7c5253bfd21..bf9066b4f481 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2593,6 +2593,11 @@ __do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx)
set_next_task(rq, p);
}

+union cpumask_rcuhead {
+ void *cpumask;
+ struct rcu_head rcu;
+};
+
/*
* Used for kthread_bind() and select_fallback_rq(), in both cases the user
* affinity (if any) should be destroyed too.
@@ -2606,7 +2611,13 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask)
};

__do_set_cpus_allowed(p, &ac);
- kfree(ac.user_mask);
+
+ /*
+ * Because this is called with p->pi_lock held, it is not possible
+ * to use kfree() here (when PREEMPT_RT=y), therefore punt to using
+ * kfree_rcu().
+ */
+ kfree_rcu((union cpumask_rcuhead *)ac.user_mask, rcu);
}

int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
@@ -8220,7 +8231,7 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
struct affinity_context ac;
struct cpumask *user_mask;
struct task_struct *p;
- int retval;
+ int retval, size;

rcu_read_lock();

@@ -8253,7 +8264,11 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
if (retval)
goto out_put_task;

- user_mask = kmalloc(cpumask_size(), GFP_KERNEL);
+ /*
+ * See do_set_cpus_allowed() for the rcu_head usage.
+ */
+ size = max_t(int, cpumask_size(), sizeof(union cpumask_rcuhead));
+ user_mask = kmalloc(size, GFP_KERNEL);
if (!user_mask) {
retval = -ENOMEM;
goto out_put_task;
--
2.31.1