Re: [PATCH][RESEND] usb: dwc3: gadget: Handle dequeuing of non queued URB gracefully
From: Lars-Peter Clausen
Date: Tue Mar 10 2020 - 10:07:10 EST
On 3/10/20 2:45 PM, Lars-Peter Clausen wrote:
On 3/10/20 2:22 PM, Ardelean, Alexandru wrote:
On Thu, 2020-01-30 at 14:02 +0200, Felipe Balbi wrote:
[External]
Hi,
Alexandru Ardelean <alexandru.ardelean@xxxxxxxxxx> writes:
From: Lars-Peter Clausen <lars@xxxxxxxxxx>
Trying to dequeue and URB that is currently not queued should be a
no-op
and be handled gracefully.
Use the list field of the URB to indicate whether it is queued or
not by
setting it to the empty list when it is not queued.
Handling this gracefully allows for race condition free synchronization
between the complete callback being called to to a completed
transfer and
trying to call usb_ep_dequeue() at the same time.
We need a little more information here. Can you further explain what
happens and how you caught this?
Apologies for the delay [of this reply].
It's been a while since this patch was created, and it was on a 4.14
kernel.
Lars was trying to fix various crashes with USB DWC3 OTG + some Xilinx
patches.
I did not track the status of the OTG stuff upstream. I think it's a
lot of
patches in the Xilinx tree.
The context has changed from 4.14 [obviously], and there were many
things that
could have influenced things.
I've been trying to RFC some of these patches now.
[ yeah I know: maybe I should have [probably] also added an RFC tag :) ]
Some of the patches [including this one] seemed to make sense, even
outside of
the context of the crashes that were happening on 4.14.
Atm, we're at 4.19 and we don't see issues, but we still have this patch.
We may drop it and see what happens.
Â\_(ã)_/Â
But in any case, it does require a bit more re-investigation.
Apologies for the noise that this patch created :)
The race condition is between a gadget calling usb_ep_dequeue() and the
driver completing the URB.
Lets say in a thread you have a reference to a in-flight URB and you
want to abort the request, e.g. because the application that sent the
request has been closed. But concurrently to that the URB is completed
by the hardware and the interrupt fires and marks the URB as complete.
Your thread is suspended while the interrupt is running, once the
interrupt has finished the thread wakes up, still has the reference to
the URB, but now it has been completed. The thread still calls
usb_ep_dequeue() though and then undefined behavior occurs.
Sorry, one quick correction. I believe the issue actually occurs when
you have more than one CPU and the thread is not suspended, while the
interrupt is running. In this case it is possible that the IRQ fires the
driver marks the URB as complete, then unlocks, the driver lock and
calls the complete callback, but before the complete callback runs the
other thread calls usb_ep_dequeue(). There is no way to protect against
this condition at the gadget level and it needs to be handled in the driver.
Basically
CPU 1 | CPU 2
--------------------------------------------------------------
URB IRQ fires | |
spin_lock(&dwc->lock); | |
| |
Driver handles completed URB | |
frees resources, etc | spin_lock(&gadget->lock) |
| usb_ep_dequeue() |
spin_unlock(&dwc->lock); | |
usb_gadget_giveback_request() | spin_lock(&dwc->lock) |
| |
Call compelte callback | Try to free URB resources |
| again => Undefined behavior |
| |
| spin_lock(&dwc->lock) |
| spin_unlock(&gadget->lock) |
spin_lock(&gadget->lock) | |
| |
Mark URB as compelte in gadget | |
spin_unlock(&gadget->lock) | |
spin_unlock(&dwc->lock) | |
spin_lock(&dwc->lock) | |
finish IRQ | |
The context in which we observed the issue is when using function fs to
create a userspace gadget and using aio_cancel() to abort a pending URB.
But really any gadget that aborts a transfer before it is completed or
before the timeout occurred can run into this issue.
- Lars