Re: skb size

Kaz Kylheku (
Wed, 20 May 1998 14:50:01 -0700 (PDT)

On Thu, 21 May 1998, Oommen Thomas wrote:

> Hi all,
> We are facing a major throughput problem on our ftp-site under
> slackware linux 2.0.30.
> Testing showed that this may be mainly due to the OS (kernel) limitations.
> UDP services like tftp/rcp as well as SMTP/UUCP give very good throughput,
> though.

Tftp is a crummy protocol; I don't see how it can give good performance
under any circumstances. :)

I believe that RCP (remote copy) works over a stream socket---you might be
thinking of RPC.

The difference is that UDP protocols tend to be based on request-response
disciplines, whereas TCP just blasts packets.

> The 'skbuff' settings in 'tcp.c' is said to be hardcoded and fails under
> heavy load. The following line is a typical case in this.
> copy = min(sk->mss - tcp_size, seglen);

I'm not sure what exactly this shows. The sk_buff identifier refers to
the fundamental network buffer data structure in Linux. On the
input side, an sk_buff structure is allocated by a network driver
when it receives a datagram, and stores the entire datagram.
The datagram is then passed to the higher level networking layers.
The buffer is linear; it's not fragmented, like the BSD mbuff.

> Is there a real problem in the kernel?
> Is this solved in newer versions?
> Is a patch available for the problem?
> The network consultant is even suggesting of replacing the 'free' linux
> with a better & powerful 'real' unix with a 'full' commercial
> implementation of TCP, which will not drop packets on heavy load.

Complete nonsense. Linux has a full implementation of TCP and a bunch
of other protocols that you don't even get with the commercial UNIX
and it's fast as well. :)

> The advice we received (which we find hard to buy) is that a free OS
> like Linux must not be used for Production servers as the kernel lacks lot
> of TCP functionalities.

Why don't you ask them to cite a list of the functionalities that it lacks?
This is a simple demand for someone who is honest, because he or she
clearly has specific functionalitities in mind. Ask them to write down a

If you replace the OS with a commercial UNIX, chances are quite high
that it will perform even worse and drop more packets, not to mention
that it will be more costly---think of the purchase price, and the
cost of support and upgrades.

If I were in your shoes, I would show a willingness to put the other
UNIX system to the test, on identical hardware. The proof of the pudding
is in the eating, so they say. Why be afraid of a little contest?
If you believe that Linux is better, you should be willing to
put it to the test.

As for your problem, you could simply be running into a memory
limitation; try installing more RAM.

It could be that kmalloc() is failing in critical code that is
requesting atomic (non blocking) memory allocation via the
GFP_ATOMIC request type.

See, when a low-level driver is doing it work at interrupt time, it cannot
call a memory management routine that wants to block a process if
no memory is available at the moment. Instead, it makes an atomic
allocation request. If the request can't be satisfied, the code
will simply balk and throw away data.

You must ensure that you have enough memory available to satisfy
these atomic requests, otherwise you get transient failures
that will cause network drivers to drop packets.

As a temporary measure, you could try recompiling the kernel
after saying ``no'' to the ``allow large windows'' option. This might
curb the memory demands of the TCP protocol so that you aren't
dropping packets.

Also, there is a way to tell the kernel to keep more memory free
by playing with the swapping thresholds by writing to
/proc/sys/vm/freepages. If you tell the kernel to swap out at an earlier
threshold, it will keep more memory available for whatever is needed.
The default settings tend to be high; that is, aggressive swapping
doesn't start until you are fairly close to running low on free
memory. This could leave you with little room for transient requests.
This setting is worth a looking into---it's not likely that the
default settings are good for your particular application.

> Any hints (to help save linux's as well as our prestige) will be greatly
> appreciated.

It surprises me that you haven't mentioned anything about your system
configuration at all. Like what hardware you are running on, what
network adapter(s) you have installed, how much memory the system
has or what other things are running on it, or how many simultaneous FTP
connections it is trying to support. If you provide these parameters,
someone will be able to provide you with more precise advice.

>From what you have said, one can only *guess* that you need more RAM.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to