Re: Kernel 3.0: Instant kernel crash when mounting CIFS (also crasheswith linux-3.1-rc2

[Steve French]

If reading files - smbclient (ftp like syntax) should be able to reach
wire speeds (assuming the server disk can keep up) and for
writing it should be similar (perhaps a little slower but it won't
use i/o sizes as large as cifs kernel client).  I would expect
smbclient to/copy from Windows to be faster than Windows mount -> Samba.

I reached near wirespeeds for GigE with cifs client (writing) to
Winodws 2003/2008/r2
and Samba - but didn't have fast enough disks to test 10GigE although I expect
very good performance with that if you have fast enough server disks (and
am willing to put performance patches in, if you detect additional
changes that we should make for 10GigE - in particular allowing
more than 50 simultaneous requests).

If the registry fix for Windows 7 is in place (or if you copy to
WIndows 2008, Windows 2003, WIndows 2008r2) -
cifs kernel client is probably slightly faster for writes than alternatives,
smbclient much faster for reads.

If going the other direction (Windows client copying to/from Samba server) -
Samba 3.6 (server) with SMB2 support
turned on - is going to be faster than most alternatives.


On Thu, Aug 18, 2011 at 12:33 PM, Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx> wrote:
>
>
> On Thu, 18 Aug 2011, Steve French wrote:
>
>> On Thu, Aug 18, 2011 at 12:16 PM, Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx>
>> wrote:
>>>
>>>
>>> On Thu, 18 Aug 2011, Jeff Layton wrote:
>>>
>>>> On Thu, 18 Aug 2011 09:15:36 -0400 (EDT)
>>>> Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, 18 Aug 2011, Jeff Layton wrote:
>>>>>
>>>>>> On Thu, 18 Aug 2011 08:22:44 -0400 (EDT)
>>>>>> Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx> wrote:
>>>>>>
>>>>>>> Justin.
>>>>>>>
>>>>>>
>>>>>> To be clear -- incoming in this case is reads or writes?
>>>>>
>>>>> Reading from the CIFS share (Windows 7).
>>>>>
>>>>>>
>>>>>> Up until 3.0 cifs.ko didn't parallelize writes from a single thread.
>>>>>> In
>>>>>> 3.0 I added a patchset to increase the allowable wsize and to allow
>>>>>> the
>>>>>> kernel to issue writes in parallel.
>>>>>
>>>>> Ahh, good to know, have not tried writes yet.
>>>>>
>>>>>>
>>>>>> Reads still suffer from the same problem however. I'm working on a
>>>>>> patchset that should do the same thing for them, but it requires a
>>>>>> fairly substantial overhaul of the receive codepaths.
>>>>>
>>>>> Ok, that explains it then, thanks.
>>>
>>>
>>> Hi,
>>>
>>> Watching the rsync, it ran for a while, then:
>>>
>>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_0.JPG": Cannot
>>> allocate memory (12)
>>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_1.JPG": Cannot
>>> allocate memory (12)
>>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_2.JPG": Cannot
>>> allocate memory (12)
>>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_0.JPG": Cannot
>>> allocate memory (12)
>>
>> When we were testing async write to Windows 7 Pavel mentioned to me
>> another WIndows 7 bug - which may be what you are hitting.
>>
>> Under stress of simultaneous operations, Windows 7 server will sometimes
>> start
>> responding with STATUS_INSUFF_SERVER_RESOURCES error code
>> (mapped to posix error ENOMEM by the Linux cifs kernel client)
>> He solved it by setting MaxWorkItems to 4096 in the Windows 7 registry.
>>
>> If anyone knows whether Microsoft has fixed this or has a bug #, let us
>> know
>> because it is easier to hit with Linux kernel 3.0 and later (to
>> Windows 7 server).
>
> Ok, aside from sharing a few files, is there any recommended protocol to
> transfer millions of files at 10GbE speeds (500MiB/s) in both directions
> between Windows and Linux?
>
> Specifically from Linux..
>
> From Windows, it achieves 500MiB/s no issues (From Linux/Samba).
>
> The other direction:
> - 30mb/s over 10GbE
> - Cannot allocate memory
>
> This should be good for moving a small subset of files, but for larger
> datasets
> it will not work.  Any other filesystems/services/etc to try?
>
> Justin.
>
>
>



-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/