Re: very poor TCP performance with 2.2.2

Mark G.Adams (madams@livepage.com)
Thu, 25 Feb 1999 22:42:23 -0500


Hi, Pete.

>Mark: can you narrow down these thousand line tracefiles into something
>just like #42 above that obviously shows the differences, just to make sure
>I'm not mistaken and there's no interaction across the conversations? And
>find out if it's really the app that's taking 50 ms vs 10 ms to generate
>the data, or if that's TCP's problem. Also you could check to see if the
>NT timeout in the 2.2 dump is always in the range 0--200 ms to confirm the
>delayed ack theory.

Alrighty. First of all, I redid the tests on both kernels. This time I ran
through them once before doing the tcp dump. This had the effect of loading
everything into the server's cache so that it can dish them out as soon as it
receives a request. This got arid of the 50 ms delay that was showing up on the
server side (down to 2 ms now). Now the NT delay is showing up clearly on it's
own (and is in the 110-190 or so range).

Here are the traces for the 2.2 and 2.0 kernel for transferring a file of size
1k. As always, server is host-210 port 3000 and client is host-232 running NT4.
Leading date is truncated for brevity.

Kernel 2.2.2-ac3
3.847389 host-232.2704 > host-210.3000: S 28542743:28542743(0) win 8192 <mss 1460> (DF) [tos 0x10]
3.847472 host-210.3000 > host-232.2704: S 986790834:986790834(0) ack 28542744 win 32120 <mss 1460> (DF)
3.847787 host-232.2704 > host-210.3000: . ack 1 win 8760 (DF) [tos 0x10]
3.848841 host-232.2704 > host-210.3000: P 1:38(37) ack 1 win 8760 (DF) [tos0x10]
3.848913 host-210.3000 > host-232.2704: . ack 38 win 32120 (DF)
3.850772 host-210.3000 > host-232.2704: P 1:157(156) ack 38 win 32120 (DF)
3.992609 host-232.2704 > host-210.3000: . ack 157 win 8604 (DF) [tos 0x10]
3.992689 host-210.3000 > host-232.2704: FP 157:1198(1041) ack 38 win 32120 (DF)
3.994086 host-232.2704 > host-210.3000: . ack 1199 win 7563 (DF) [tos 0x10]
3.994266 host-232.2704 > host-210.3000: F 38:38(0) ack 1199 win 7563 (DF) [tos 0x10]
3.994326 host-210.3000 > host-232.2704: . ack 39 win 32120 (DF)

Kernel 2.0.36
7.358420 host-232.3010 > host-210.3000: S 28545008:28545008(0) win 8192 <mss 1460> (DF) [tos 0x10]
7.358420 host-210.3000 > host-232.3010: S 3671755690:3671755690(0) ack 28545009 win 32736 <mss 1460>
7.358420 host-232.3010 > host-210.3000: . ack 1 win 8760 (DF) [tos 0x10]
7.358420 host-232.3010 > host-210.3000: P 1:38(37) ack 1 win 8760 (DF) [tos 0x10]
7.368420 host-210.3000 > host-232.3010: P 1:157(156) ack 38 win 32736 (DF) [tos 0x10]
7.368420 host-210.3000 > host-232.3010: P 157:1198(1041) ack 38 win 32736 (DF) [tos 0x10]
7.368420 host-210.3000 > host-232.3010: F 1198:1198(0) ack 38 win 32736 [tos 0x10]
7.368420 host-232.3010 > host-210.3000: . ack 1199 win 7563 (DF) [tos 0x10]
7.368420 host-232.3010 > host-210.3000: F 38:38(0) ack 1199 win 7563 (DF) [tos 0x10]
7.368420 host-210.3000 > host-232.3010: . ack 39 win 32735 (DF) [tos 0x10]

Note how in the version for the 2.2 kernel, the NT timeout is present (142 ms)
but not being triggered in the trace for the 2.0 kernel.

When the file size is bumped to 2k, the situation is even more pronounced since
the NT timeout is hit 2x.

Kernel 2.2.2-ac3
5.799123 host-232.2714 > host-210.3000: S 28542816:28542816(0) win 8192 <mss 1460> (DF) [tos 0x10]
5.799216 host-210.3000 > host-232.2714: S 985258180:985258180(0) ack 28542817 win 32120 <mss 1460> (DF)
5.799534 host-232.2714 > host-210.3000: . ack 1 win 8760 (DF) [tos 0x10]
5.800406 host-232.2714 > host-210.3000: P 1:38(37) ack 1 win 8760 (DF) [tos 0x10]
5.800484 host-210.3000 > host-232.2714: . ack 38 win 32120 (DF)
5.802366 host-210.3000 > host-232.2714: P 1:157(156) ack 38 win 32120 (DF)
5.995680 host-232.2714 > host-210.3000: . ack 157 win 8604 (DF) [tos 0x10]
5.995765 host-210.3000 > host-232.2714: . 157:1617(1460) ack 38 win 32120 (DF)
6.195980 host-232.2714 > host-210.3000: . ack 1617 win 8760 (DF) [tos 0x10]
6.196054 host-210.3000 > host-232.2714: FP 1617:2157(540) ack 38 win 32120 (DF)
6.196941 host-232.2714 > host-210.3000: . ack 2158 win 8220 (DF) [tos 0x10]
6.197105 host-232.2714 > host-210.3000: F 38:38(0) ack 2158 win 8220 (DF) [tos 0x10]
6.197164 host-210.3000 > host-232.2714: . ack 39 win 32120 (DF)

Kernel 2.0.36
7.438420 host-232.3020 > host-210.3000: S 28545083:28545083(0) win 8192 <mss 1460> (DF) [tos 0x10]
7.438420 host-210.3000 > host-232.3020: S 2532093908:2532093908(0) ack 28545084 win 32736 <mss 1460>
7.438420 host-232.3020 > host-210.3000: . ack 1 win 8760 (DF) [tos 0x10]
7.438420 host-232.3020 > host-210.3000: P 1:38(37) ack 1 win 8760 (DF) [tos 0x10]
7.448420 host-210.3000 > host-232.3020: P 1:157(156) ack 38 win 32736 (DF) [tos 0x10]
7.448420 host-210.3000 > host-232.3020: . 157:1617(1460) ack 38 win 32736 (DF) [tos 0x10]
7.448420 host-232.3020 > host-210.3000: . ack 1617 win 8760 (DF) [tos 0x10]
7.448420 host-210.3000 > host-232.3020: P 1617:2157(540) ack 38 win 32736 (DF) [tos 0x10]
7.448420 host-210.3000 > host-232.3020: F 2157:2157(0) ack 38 win 32736 [tos 0x10]
7.448420 host-232.3020 > host-210.3000: . ack 2158 win 8220 (DF) [tos 0x10]
7.448420 host-232.3020 > host-210.3000: F 38:38(0) ack 2158 win 8220 (DF) [tos 0x10]
7.448420 host-210.3000 > host-232.3020: . ack 39 win 32735 (DF) [tos 0x10]

Finally, here's a trace with filesize 8k.

Kernel 2.2.2-ac3
1.811370 host-232.2734 > host-210.3000: S 28543007:28543007(0) win 8192 <mss 1460> (DF) [tos 0x10]
1.811470 host-210.3000 > host-232.2734: S 982474832:982474832(0) ack 28543008 win 32120 <mss 1460> (DF)
1.811789 host-232.2734 > host-210.3000: . ack 1 win 8760 (DF) [tos 0x10]
1.812662 host-232.2734 > host-210.3000: P 1:38(37) ack 1 win 8760 (DF) [tos 0x10]
1.812735 host-210.3000 > host-232.2734: . ack 38 win 32120 (DF)
1.814639 host-210.3000 > host-232.2734: P 1:157(156) ack 38 win 32120 (DF)
2.004872 host-232.2734 > host-210.3000: . ack 157 win 8604 (DF) [tos 0x10]
2.004952 host-210.3000 > host-232.2734: . 157:1617(1460) ack 38 win 32120 (DF)
2.005117 host-210.3000 > host-232.2734: . 1617:3077(1460) ack 38 win 32120 (DF)
2.008010 host-232.2734 > host-210.3000: . ack 3077 win 8760 (DF) [tos 0x10]
2.008082 host-210.3000 > host-232.2734: . 3077:4537(1460) ack 38 win 32120 (DF)
2.008244 host-210.3000 > host-232.2734: . 4537:5997(1460) ack 38 win 32120 (DF)
2.009482 host-210.3000 > host-232.2734: . 5997:7457(1460) ack 38 win 32120 (DF)
2.012051 host-232.2734 > host-210.3000: . ack 5997 win 8760 (DF) [tos 0x10]
2.205181 host-232.2734 > host-210.3000: . ack 7457 win 8760 (DF) [tos 0x10]
2.205258 host-210.3000 > host-232.2734: FP 7457:7917(460) ack 38 win 32120 (DF)
2.206053 host-232.2734 > host-210.3000: . ack 7918 win 8300 (DF) [tos 0x10]
2.206210 host-232.2734 > host-210.3000: F 38:38(0) ack 7918 win 8300 (DF) [tos 0x10]
2.206267 host-210.3000 > host-232.2734: . ack 39 win 32120 (DF)

Kernel 2.0.36
7.638420 host-232.3040 > host-210.3000: S 28545222:28545222(0) win 8192 <mss 1460> (DF) [tos 0x10]
7.638420 host-210.3000 > host-232.3040: S 395849035:395849035(0) ack 28545223 win 32736 <mss 1460>
7.638420 host-232.3040 > host-210.3000: . ack 1 win 8760 (DF) [tos 0x10]
7.638420 host-232.3040 > host-210.3000: P 1:38(37) ack 1 win 8760 (DF) [tos 0x10]
7.638420 host-210.3000 > host-232.3040: P 1:157(156) ack 38 win 32736 (DF) [tos 0x10]
7.638420 host-210.3000 > host-232.3040: . 157:1617(1460) ack 38 win 32736 (DF) [tos 0x10]
7.638420 host-232.3040 > host-210.3000: . ack 1617 win 8760 (DF) [tos 0x10]
7.638420 host-210.3000 > host-232.3040: . 1617:3077(1460) ack 38 win 32736 [tos 0x10]
7.638420 host-210.3000 > host-232.3040: . 3077:4537(1460) ack 38 win 32736 [tos 0x10]
7.638420 host-210.3000 > host-232.3040: . 4537:5997(1460) ack 38 win 32736 [tos 0x10]
7.648420 host-232.3040 > host-210.3000: . ack 4537 win 8760 (DF) [tos 0x10]
7.648420 host-210.3000 > host-232.3040: . 5997:7457(1460) ack 38 win 32736 [tos 0x10]
7.648420 host-210.3000 > host-232.3040: P 7457:7917(460) ack 38 win 32736 (DF) [tos 0x10]
7.648420 host-210.3000 > host-232.3040: F 7917:7917(0) ack 38 win 32736 [tos 0x10]
7.648420 host-232.3040 > host-210.3000: . ack 7457 win 8760 (DF) [tos 0x10]
7.648420 host-232.3040 > host-210.3000: . ack 7918 win 8300 (DF) [tos 0x10]
7.648420 host-232.3040 > host-210.3000: F 38:38(0) ack 7918 win 8300 (DF) [tos 0x10]
7.648420 host-210.3000 > host-232.3040: . ack 39 win 32735 (DF) [tos 0x10]

Once again, there are 2 NT delays here (190 and 193 ms) with the 2.2 trace
which aren't present in the 2.0 trace.

As always, let me know if you need any more info.

//Mark

--
Mark G. Adams
Software Developer
LivePage Corporation
Waterloo, Ontario, Canada
Email: madams@livepage.com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/