Re: [PATCH v3 3/3] md: protect md_thread with rcu

[Logan Gunthorpe]

On 2023-03-30 14:20, Yu Kuai wrote:
> From: Yu Kuai <yukuai3@xxxxxxxxxx>
> 
> Our test reports a uaf for 'mddev->sync_thread':
> 
> T1                      T2
> md_start_sync
>  md_register_thread
>  // mddev->sync_thread is set
> 			raid1d
> 			 md_check_recovery
> 			  md_reap_sync_thread
> 			   md_unregister_thread
> 			    kfree
> 
>  md_wakeup_thread
>   wake_up
>   ->sync_thread was freed
> 
> Root cause is that there is a small windown between register thread and
> wake up thread, where the thread can be freed concurrently.
> 
> Currently, a global spinlock 'pers_lock' is borrowed to protect
> 'mddev->thread', this problem can be fixed likewise, however, there might
> be similar problem for other md_thread, and I really don't like the idea to
> borrow a global lock.
> 
> This patch protect md_thread with rcu.
> 
> Signed-off-by: Yu Kuai <yukuai3@xxxxxxxxxx>
> ---
>  drivers/md/md.c | 32 ++++++++++----------------------
>  1 file changed, 10 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 9e80c5491c9a..161231e01faa 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -70,11 +70,7 @@
>  #include "md-bitmap.h"
>  #include "md-cluster.h"
>  
> -/* pers_list is a list of registered personalities protected
> - * by pers_lock.
> - * pers_lock does extra service to protect accesses to
> - * mddev->thread when the mutex cannot be held.
> - */
> +/* pers_list is a list of registered personalities protected by pers_lock. */
>  static LIST_HEAD(pers_list);
>  static DEFINE_SPINLOCK(pers_lock);
>  
> @@ -802,13 +798,8 @@ void mddev_unlock(struct mddev *mddev)
>  	} else
>  		mutex_unlock(&mddev->reconfig_mutex);
>  
> -	/* As we've dropped the mutex we need a spinlock to
> -	 * make sure the thread doesn't disappear
> -	 */
> -	spin_lock(&pers_lock);
>  	md_wakeup_thread(&mddev->thread);
>  	wake_up(&mddev->sb_wait);
> -	spin_unlock(&pers_lock);
>  }
>  EXPORT_SYMBOL_GPL(mddev_unlock);
>  
> @@ -7921,13 +7912,16 @@ static int md_thread(void *arg)
>  
>  void md_wakeup_thread(struct md_thread **threadp)
>  {
> -	struct md_thread *thread = *threadp;
> +	struct md_thread *thread;
>  
> +	rcu_read_lock();
> +	thread = rcu_dereference(*threadp);

A couple points:

I don't think we need a double pointer here. rcu_dereference() doesn't
actually do anything but annotate the fact that we are accessing a
pointer protected by rcu. It does require annotations on that pointer
(__rcu) which is checked by sparse (I suspect this patch will produce a
lot of sparse errors from kbuild bot).

I think all we need is:

void md_wakeup_thread(struct md_thread __rcu *rthread)
{
	struct md_thread *thread;
   	
	rcu_read_lock();
	thread = rcu_dereference(rthread);
	...
	rcu_read_unlock();
	
}

The __rcu annotation will have to be added to all the pointers this
function is called on as well as to md_register_thread() and
md_unregister_thread(). And anything else that uses those pointers.
Running sparse on the code and eliminating all new errors for the patch
is important.

Thanks,

Logan