Re: [RFC][PATCH] usb: dwc3: usb: dwc3: Force stop EP0 transfers during pullup disable

[Thinh Nguyen]

Thinh Nguyen wrote:
> Wesley Cheng wrote:
>> Hi Thinh,
>>
>> On 8/13/2021 4:21 PM, Thinh Nguyen wrote:
>>> Wesley Cheng wrote:
>>>> Hi Thinh,
>>>>
>>>> On 8/12/2021 2:31 PM, Thinh Nguyen wrote:
>>>>> Wesley Cheng wrote:
>>>>>> Hi Thinh,
>>>>>>
>>>>>> On 8/11/2021 5:47 PM, Thinh Nguyen wrote:
>>>>>>> Wesley Cheng wrote:
>>>>>>>> During a USB cable disconnect, or soft disconnect scenario, a pending
>>>>>>>> SETUP transaction may not be completed, leading to the following
>>>>>>>> error:
>>>>>>>>
>>>>>>>>     dwc3 a600000.dwc3: timed out waiting for SETUP phase
>>>>>>>
>>>>>>> How could it be a case of cable disconnect? The pullup(0) only applies
>>>>>>> for soft-disconnect scenario. If the device initiated a disconnect, then
>>>>>>
>>>>>> Thanks for the response.  I guess this is specific for some use cases,
>>>>>> but some applications such as ADB will close the FFS EP files after it
>>>>>> gets the disconnection event, leading to this pullup disable as well.
>>>>>> So its specific to that particular use case.
>>>>>
>>>>> Does that mean that the ADB application won't expect a connection until
>>>>> user intervention or some other notification to do pullup(1)?
>>>>>
>>>>
>>>> Yes, correct.  The Android USB framework will trigger the pullup(1) again.
>>>>
>>>>>>
>>>>>>> the driver should wait for the control request to complete. If it times
>>>>>>> out, something is already wrong here. The programming guide only
>>>>>>> mentions that we should wait for completion, but it doesn't say about
>>>>>>> recovery in a case of hung transfer. I need to check internally but it
>>>>>>> should be safe to issue End Transfer.
>>>>>>>
>>>>>>
>>>>>> Yes, what I did was modify a device running the Linux XHCI stack w/o
>>>>>> reading/sending out the SETUP DATA phase, so that on the device end we'd
>>>>>> always run into that situation where there's still a pending EP0 TRB queued.
>>>>>
>>>>> Is this only for validation purpose?
>>>>>
>>>>
>>>> Yes, just to help verify the fix by injecting the error condition.
>>>
>>> Just want to clarify, so you didn't really run into this problem but
>>> want to fix the recovery path right?
>>>
>> No, we ran into this issue, but it was only seen in our mass device
>> testing.  I only made the changes to simulate the issue, so then I
>> wouldn't need to have to wait for the test results.
>>
>> However, we did get a mass device test run in w/ the changes, and they
>> were unable to reproduce the same issues we've been seeing, so this does
>> fix an actual issue.
> 
> Ok
> 
>>>>
>>>>>>
>>>>>> We're running multiple devices with this fix as well, and doing device
>>>>>> initiated disconnect.
>>>>>>
>>>>>>>>
>>>>>>>> If this occurs, then the entire pullup disable routine is skipped and
>>>>>>>> proper cleanup and halting of the controller does not complete.
>>>>>>>> Instead of returning an error (which is ignored from the UDC
>>>>>>>> perspective), do what is mentioned in the comments and force the
>>>>>>>> transaction to complete and put the ep0state back to the SETUP phase.
>>>>>>>>
>>>>>>>> Signed-off-by: Wesley Cheng <wcheng@xxxxxxxxxxxxxx>
>>>>>>>> ---
>>>>>>>>  drivers/usb/dwc3/ep0.c    | 4 ++--
>>>>>>>>  drivers/usb/dwc3/gadget.c | 6 +++++-
>>>>>>>>  drivers/usb/dwc3/gadget.h | 3 +++
>>>>>>>>  3 files changed, 10 insertions(+), 3 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/usb/dwc3/ep0.c b/drivers/usb/dwc3/ep0.c
>>>>>>>> index 6587394..abfc42b 100644
>>>>>>>> --- a/drivers/usb/dwc3/ep0.c
>>>>>>>> +++ b/drivers/usb/dwc3/ep0.c
>>>>>>>> @@ -218,7 +218,7 @@ int dwc3_gadget_ep0_queue(struct usb_ep *ep, struct usb_request *request,
>>>>>>>>  	return ret;
>>>>>>>>  }
>>>>>>>>  
>>>>>>>> -static void dwc3_ep0_stall_and_restart(struct dwc3 *dwc)
>>>>>>>> +void dwc3_ep0_stall_and_restart(struct dwc3 *dwc)
>>>>>>>>  {
>>>>>>>>  	struct dwc3_ep		*dep;
>>>>>>>>  
>>>>>>>> @@ -1073,7 +1073,7 @@ void dwc3_ep0_send_delayed_status(struct dwc3 *dwc)
>>>>>>>>  	__dwc3_ep0_do_control_status(dwc, dwc->eps[direction]);
>>>>>>>>  }
>>>>>>>>  
>>>>>>>> -static void dwc3_ep0_end_control_data(struct dwc3 *dwc, struct dwc3_ep *dep)
>>>>>>>> +void dwc3_ep0_end_control_data(struct dwc3 *dwc, struct dwc3_ep *dep)
>>>>>>>>  {
>>>>>>>>  	struct dwc3_gadget_ep_cmd_params params;
>>>>>>>>  	u32			cmd;
>>>>>>>> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
>>>>>>>> index 54c5a08..a0e2e4d 100644
>>>>>>>> --- a/drivers/usb/dwc3/gadget.c
>>>>>>>> +++ b/drivers/usb/dwc3/gadget.c
>>>>>>>> @@ -2437,7 +2437,11 @@ static int dwc3_gadget_pullup(struct usb_gadget *g, int is_on)
>>>>>>>>  				msecs_to_jiffies(DWC3_PULL_UP_TIMEOUT));
>>>>>>>>  		if (ret == 0) {
>>>>>>>>  			dev_err(dwc->dev, "timed out waiting for SETUP phase\n");
>>>>>>>> -			return -ETIMEDOUT;
>>>>>>>> +			spin_lock_irqsave(&dwc->lock, flags);
>>>>>>>> +			dwc3_ep0_end_control_data(dwc, dwc->eps[0]);
>>>>>>>> +			dwc3_ep0_end_control_data(dwc, dwc->eps[1]);
>>>>>>>
>>>>>>> End transfer command takes time, need to wait for it to complete before
>>>>>>> issuing Start transfer again. Also, why restart again when it's about to
>>>>>>> be disconnected.
>>>>>>
>>>>>> I can try without restarting it again, and see if that works.  Instead
>>>>>> of waiting for the command complete event, can we set the ForceRM bit,
>>>>>> similar to what we do for dwc3_remove_requests()?
>>>>>>
>>>>>
>>>>> ForceRM=1 means that the controller will ignore updating the TRBs
>>>>> (including not clearing the HWO and remain transfer size). The driver
>>>>> still needs to wait for the command to complete before issuing Start
>>>>> Transfer command. Otherwise Start Transfer won't go through. If we know
>>>>> that we're not going to issue Start Transfer any time soon, then we may
>>>>> be able to get away with ignoring End Transfer command completion.
>>>>>
>>>>
>>>> I see.  Currently, in the place that we do use
>>>> dwc3_ep0_end_control_data(), its followed by
>>>> dwc3_ep0_stall_and_restart() which would execute start transfer.  For
>>>
>>> That doesn't look right. You can try to see if it can recover from a
>>> control write request. Often time we do control read and not write.
>>> (i.e. try to End Transfer and immediately Start Transfer on the same
>>> direction control endpoint).
>>>
>> OK, I can try, but just to clarify, I was referring to how it was being
>> done in:
>>
>> static void dwc3_ep0_xfernotready(struct dwc3 *dwc,
>> 		const struct dwc3_event_depevt *event)
>> {
>> ...
>> 		if (dwc->ep0_expect_in != event->endpoint_number) {
>> 			struct dwc3_ep	*dep = dwc->eps[dwc->ep0_expect_in];
>>
>> 			dev_err(dwc->dev, "unexpected direction for Data Phase\n");
>> 			dwc3_ep0_end_control_data(dwc, dep);
>> 			dwc3_ep0_stall_and_restart(dwc);
>> 			return;
>> 		}
>>

Looking at this snippet again, it looks wrong. For control write
unexpected direction, if the driver hasn't setup and started the DATA
phase yet, then it's fine, but there is a problem if it did.

Since dwc3_ep0_end_control_data() doesn't issue End Transfer command to
ep0 due to the resource_index check, it doesn't follow the control
transfer flow model in the programming guide. This may cause
dwc3_ep0_stall_and_restart() to overwrite the TRBs for the DATA phase
with SETUP stage. Also, if the ep0 is already started, the driver won't
issue Start Transfer command again.

This issue is unlikely to occur unless we see a misbehave host for
control write request. Regardless, we need to fix this. I may need some
time before I can create a patch and test it. If you or anyone is up to
take this on, it'd be highly appreciated.

If I missed anything, do let me know.

Thanks,
Thinh