Re: [PATCH v2 2/2] usbip: Implement SG support to vhci

[Suwan Kim]

On Fri, Aug 02, 2019 at 07:33:59AM -0600, shuah wrote:
> On 8/2/19 1:41 AM, Suwan Kim wrote:
> > On Thu, Aug 01, 2019 at 08:03:59AM -0600, shuah wrote:
> > > On 8/1/19 12:38 AM, Suwan Kim wrote:
> > > > On Mon, Jul 29, 2019 at 10:32:31AM -0600, shuah wrote:
> > > > > On 7/29/19 8:52 AM, Suwan Kim wrote:
> > > > > > Hi Shuah,
> > > > > > 
> > > > > > On Tue, Jul 23, 2019 at 06:21:53PM -0600, shuah wrote:
> > > > > > > Hi Suwan,
> > > > > > > 
> > > > > > > On 7/5/19 10:43 AM, Suwan Kim wrote:
> > > > > > > > There are bugs on vhci with usb 3.0 storage device. Originally, vhci
> > > > > > > > doesn't supported SG, so USB storage driver on vhci breaks SG list
> > > > > > > > into multiple URBs and it causes error that a transfer got terminated
> > > > > > > > too early because the transfer length for one of the URBs was not
> > > > > > > > divisible by the maxpacket size.
> > > > > > > > 
> > > > > > > > In this patch, vhci basically support SG and it sends each SG list
> > > > > > > > entry to the stub driver. Then, the stub driver sees the total length
> > > > > > > > of the buffer and allocates SG table and pages according to the total
> > > > > > > > buffer length calling sgl_alloc(). After the stub driver receives
> > > > > > > > completed URB, it again sends each SG list entry to vhci.
> > > > > > > > 
> > > > > > > > If HCD of the server doesn't support SG, the stub driver breaks a
> > > > > > > > single SG reqeust into several URBs and submit them to the server's
> > > > > > > > HCD. When all the split URBs are completed, the stub driver
> > > > > > > > reassembles the URBs into a single return command and sends it to
> > > > > > > > vhci.
> > > > > > > > 
> > > > > > > > Signed-off-by: Suwan Kim <suwan.kim027@xxxxxxxxx>
> > > > > > > > ---
> > > > > > > >      drivers/usb/usbip/stub.h         |   7 +-
> > > > > > > >      drivers/usb/usbip/stub_main.c    |  52 +++++---
> > > > > > > >      drivers/usb/usbip/stub_rx.c      | 207 ++++++++++++++++++++++---------
> > > > > > > >      drivers/usb/usbip/stub_tx.c      | 108 +++++++++++-----
> > > > > > > >      drivers/usb/usbip/usbip_common.c |  60 +++++++-- >   drivers/usb/usbip/vhci_hcd.c     |  10 +-
> > > > > > > >      drivers/usb/usbip/vhci_tx.c      |  49 ++++++--
> > > > > > > >      7 files changed, 372 insertions(+), 121 deletions(-)
> > > > > > > 
> > > > > > > While you are working on v3 to fix chekpatch and other issues
> > > > > > > I pointed out, I have more for you.
> > > > > > > 
> > > > > > > What happens when you have mismatched server and client side?
> > > > > > > i.e stub does and vhci doesn't and vice versa.
> > > > > > > 
> > > > > > > Make sure to run checkpatch. Also check spelling errors in
> > > > > > > comments and your commit log.
> > > > > > > 
> > > > > > > I am not sure if your eeror paths are correct. Run usbip_test.sh
> > > > > > > 
> > > > > > > tools/testing/selftests/drivers/usb/usbip
> > > > > > 
> > > > > > I don't know what mismatch you mentioned. Are you saying
> > > > > > "match busid table" at the end of usbip_test.sh?
> > > > > > How does it relate to SG support of this patch?
> > > > > > Could you tell me more about the problem situation?
> > > > > > 
> > > > > 
> > > > > What happens when usbip_host is running a kernel without the sg
> > > > > support and vhci_hcd does? Just make sure this works with the
> > > > > checks for sg support status as a one of your tests for this
> > > > > sg feature.
> > > > 
> > > > Now I understand. Thanks for the details!
> > > > As a result of testing, in the situation where vhci supports SG,
> > > > but stub does not, or vice versa, usbip works normally. Moreover,
> > > > because there is no protocol modification, there is no problem in
> > > > communication between server and client even if the one has a kernel
> > > > without SG support.
> > > > 
> > > > In the case of vhci supports SG and stub doesn't, because vhci sends
> > > > only the total length of the buffer to stub as it did before the
> > > > patch applied, stub only needs to allocate the required length of
> > > > buffers regardless of whether vhci supports SG or not.
> > > > 
> > > > If stub needs to send data buffer to vhci because of IN pipe, stub
> > > > also sends only total length of buffer as metadata and then send real
> > > > data as vhci does. Then vhci receive data from stub and store it to
> > > > the corresponding buffer of SG list entry.
> > > > 
> > > > In the case of stub that supports SG, if SG buffer is requested by
> > > > vhci, buffer is allocated by sgl_alloc(). However, stub that does
> > > > not support SG allocates buffer using only kmalloc(). Therefore, if
> > > > vhci supports SG and stub doesn't, stub has to allocate buffer with
> > > > kmalloc() as much as the total length of SG buffer which is quite
> > > > huge when vhci sends SG request, so it has big overhead in buffer
> > > > allocation.
> > > > 
> > > > And for the case of stub supports SG and vhci doesn't, since the
> > > > USB storage driver checks that vhci doesn't support SG and sends
> > > > the request to stub by splitting the SG list into multiple URBs,
> > > > stub allocates a buffer with kmalloc() as it did before this patch.
> > > > 
> > > > Therefore, everything works normally in a mismatch situation.
> > > 
> > > Looking for you add a test case for that. Please include this
> > > in the commit log.
> > 
> > I'm confused about the test case. Do I add the test case for each
> > SG support status of vhci_hcd and usbip_host in usbip_test.sh?
> > Or, do I implement the test logic in vhci_hcd code that asks if
> > usbip_host supports SG when attaching a remote device?
> > I'm sorry but I don't know what exactly you want to add.
> > 
> 
> What I am asking you do is:
> 
> 1. test with mismatched host and client.

I already tested this case with two different machines. One is the
5.2-rc6 kernel with SG patch and the other is default fedora kernel
version 5.1.20-200.fc29.x86_64. As I said, I run vhci_hcd with SG
and usbip_host without SG and also vice versa.

I tested it with Sandisk USB 3.0 storage deivce and tested read and
write operation which transfer files whose size is over 1G like movie
file. All the opreations work well.

> 2. run usbip_test.sh

One thing goes wrong. But others except the one worked as expected.

==============================================================
List imported devices - expect to see imported devices
Imported USB devices
====================
Port 08: <Port in Use> at Super Speed(5000Mbps)
       SanDisk Corp. : Ultra Flair (0781:5591)
       4-1 -> usbip://localhost:3240/2-6
           -> remote bus/dev 002/003
==============================================================
Import devices from localhost - expect already imported messages
usbip: error: Attach Request for 2-6 failed - Device busy (exported)

==============================================================
Un-import devices
usbip: info: Port 0 is already detached!

usbip: info: Port 1 is already detached!

==============================================================
List imported devices - expect to see none
Imported USB devices
====================
Port 08: <Port in Use> at Super Speed(5000Mbps)
       SanDisk Corp. : Ultra Flair (0781:5591)
       4-1 -> usbip://localhost:3240/2-6
           -> remote bus/dev 002/003
==============================================================

My SanDisk device is bound port 8 but the test script detachs only
port 0 and 1. This error is not related with SG patch but we have to
modifiy the test script to try to detach all the ports.

> These two are separate tests. I am not asking you to add any tests.
> If you want to add it, I am not going say no :)
> 
> How close are you sending the patch?

My work is done. I will send it now.

Regards
Suwan Kim