In vudc_probe, it calls init_vudc_hw, which bound &udc->timer with v_timer.
When it calls usbip_sockfd_store, it will call v_start_timer to start the
timer work.
When we call vudc_remove to remove the driver, theremay be a sequence as
follows:
Fix it by shutdown the timer work before cleanup in vudc_remove.