Re: [PATCH] fs: Fix freeze_bdev()/thaw_bdev() accounting of bd_fsfreeze_sb

[Satya Tangirala]

On Thu, Jan 07, 2021 at 11:27:37AM -0500, Bob Peterson wrote:
> ----- Original Message -----
> > Can someone pick this up?  Maybe through Jens' block tree as that is
> > where my commit this is fixing up came from.
> Christoph and Al,
> 
> Here is my version:
> 
> Bob Peterson
> 
> fs: fix freeze count problem in freeze_bdev
> 
> Before this patch, if you tried to freeze a device (function freeze_bdev)
> while it was being unmounted, it would get NULL back from get_active_super
> and correctly bypass the freeze calls. Unfortunately, it forgot to decrement
> its bd_fsfreeze_count. Subsequent calls to device thaw (thaw_bdev) would
> see the non-zero bd_fsfreeze_count and assume the bd_fsfreeze_sb value was
> still valid. That's not a safe assumption and resulted in use-after-free,
> which often caused fatal kernel errors like: "unable to handle page fault
> for address."
> 
> This patch adds the necessary decrement of bd_fsfreeze_count for that
> error path. It also adds code to set the bd_fsfreeze_sb to NULL when the
> last reference is reached in thaw_bdev.
> 
> Reviewed-by: Bob Peterson <rpeterso@xxxxxxxxxx>
> ---
>  fs/block_dev.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/block_dev.c b/fs/block_dev.c
> index 9e56ee1f2652..c6daf7d12546 100644
> --- a/fs/block_dev.c
> +++ b/fs/block_dev.c
> @@ -555,8 +555,10 @@ int freeze_bdev(struct block_device *bdev)
>  		goto done;
>  
>  	sb = get_active_super(bdev);
> -	if (!sb)
> +	if (!sb) {
> +		bdev->bd_fsfreeze_count--;
>  		goto sync;
> +	}
>  	if (sb->s_op->freeze_super)
>  		error = sb->s_op->freeze_super(sb);
>  	else
> @@ -600,6 +602,7 @@ int thaw_bdev(struct block_device *bdev)
>  	if (!sb)
>  		goto out;
>  
> +	bdev->bd_fsfreeze_sb = NULL;
This causes bdev->bd_fsfreeze_sb to be set to NULL even if the call to
thaw_super right after this line fail. So if a caller tries to call
thaw_bdev() again after receiving such an error, that next call won't even
try to call thaw_super(). Is that what we want here?  (I don't know much
about this code, but from a cursory glance I think this difference is
visible to emergency_thaw_bdev() in fs/buffer.c)

In my version of the patch, I set bdev->bd_fsfreeze_sb to NULL only
*after* we check that the call to thaw_super() succeeded to avoid this.
>  	if (sb->s_op->thaw_super)
>  		error = sb->s_op->thaw_super(sb);
>  	else
>
In another mail, you mentioned
> I wrote this patch to fix the freeze/thaw device problem before I saw
> the patch "fs: Fix freeze_bdev()/thaw_bdev() accounting of bd_fsfreeze_sb"
> from Satya Tangirala. That one, however, does not fix the bd_freeze_count
> problem and this patch does.
Thanks a lot for investigating the bug and the patch I sent :)
Was there actually an issue with that patch I sent? As you said, the bug
is very reproduceable on master with generic/085. But with my patch, I
don't see any issues even after having run the test many, many times
(admittedly, I tested it on f2fs and ext4 rather than gfs2, but I don't
think that should cause much differences). Did you have a test case that
actually causes a failure with my patch?

The only two differences between the patch I sent and this patch are

1) The point at which the bd_fsfreeze_bd is set to NULL in thaw_bdev(), as
I mentioned above already.

2) Whether or not to decrement bd_fsfreeze_count when we get a NULL from
get_active_super() in freeze_bdev() - I don't do this in my patch.

I think setting bd_fsfreeze_sb to NULL in thaw_bdev (in either the place
your patch does it or my patch does it) is enough to fix the bug w.r.t the
use-after-free. Fwiw, I do think it should be set to NULL after we check for
all the errors though.

I think the second difference (decrementing bd_fsfreeze_count when
get_active_super() returns NULL) doesn't change anything w.r.t the
use-after-free. It does however, change the behaviour of the function
slightly, and it might be caller visible (because from a cursory glance, it
looks like we're reading the bd_fsfreeze_count from some other places like
fs/super.c). Even before 040f04bd2e82, the code wouldn't decrement
bd_fsfreeze_count when get_active_super() returned NULL - so is this change
in behaviour intentional? And if so, maybe it should go in a separate
patch?