Re: [RESEND PATCH V5 6/7] ublk_drv: add START_USER_RECOVERY and END_USER_RECOVERY support

[Ziyang Zhang]

On 2022/9/23 14:15, ZiyangZhang wrote:
> START_USER_RECOVERY and END_USER_RECOVERY are two new control commands
> to support user recovery feature.
> 
> After a crash, user should send START_USER_RECOVERY, it will:
> (1) check if (a)current ublk_device is UBLK_S_DEV_QUIESCED which was
>     set by quiesce_work and (b)chardev is released
> (2) reinit all ubqs, including:
>     (a) put the task_struct and reset ->ubq_daemon to NULL.
>     (b) reset all ublk_io.
> (3) reset ub->mm to NULL.
> 
> Then, user should start a new process and send FETCH_REQ on each
> ubq_daemon.
> 
> Finally, user should send END_USER_RECOVERY, it will:
> (1) wait for all new ubq_daemons getting ready.
> (2) update ublksrv_pid
> (3) unquiesce the request queue and expect incoming ublk_queue_rq()
> (4) convert ub's state to UBLK_S_DEV_LIVE
> 
> Note: we can handle STOP_DEV between START_USER_RECOVERY and
> END_USER_RECOVERY. This is helpful to users who cannot start new process
> after sending START_USER_RECOVERY ctrl-cmd.
> 
> Signed-off-by: ZiyangZhang <ZiyangZhang@xxxxxxxxxxxxxxxxx>

Let me explain why we still need two ctrl-cmds: START_USER_RECOVERY
and END_USER_RECOVERY:

(1) They are easy to use and understand. Just like calling
    a)ADD_DEV, b)start daemon c)START_DEV, we can recover a device by
    a)START_USER_RECOVERY b)start new daemon c)END_USER_RECOVERY.
    IMO, START_USER_RECOVERY can guide user whether to start a new daemon.
    Without it, user must directly start a new daemon. He may fail here
    because /dev/ublkc* is not released. So a retry is necessary here.
    But I think that let the user retry by sending START_USER_RECOVERY
    instead of opening /dev/ublkc* is more reasonable.

(2) Handling put_task_struct(ubq_daemon) is hard in ublk_ch_release().
    Assume all ioucmds have been issued back to userspace and a crash happens,
    ublk_ch_release() can be called immediately here and ubq_daemon is
    freed. But monoitor_work may be running now. Dealing with UAF on
    ubq_daemon in monitor_work may be difficult. But handling
    put_task_struct(ubq_daemon) in START_USER_RECOVERY solves the problem
    because monitor_work is sure to be canceled. Besides, here is no race
    with ublk_deinit_queue() since it cannot be called if ub's state is
    QUIESCED.

Regards,
Zhang