Possible TCP BUG. - looping retries.

Steve Hanselman (shanselman@brendata.co.uk)
Tue, 12 Jan 1999 17:21:23 -0000


I have attached a network trace of a conversation between one of our Linux
machines (2.0 pre4, but in fact almost every release since 2.1.100) and a
printserver here at Brendata.

Up to (and including) version 2.1.90 this worked fine. Some point after
2.1.90 it has died. If I reboot back to 2.1.90 it is fine. If you give me
some pointers I'm happy to investigate whatever. Later today, I intend to
pull the ipv4 directory from 2.1.90 and compare against 2.2.0pre5 and see
whether there is anything obvious.

(apologies about the size but I think that it gives all the information
required).

It appears that the stack effectively gets blocked just for this one
conversation?

Here is my annotation:
Packet 0 - Linux opens connection (Syn)
Packet 1 - pserv acks the open (SynAck)
Packet 2 - Linux acks the synack
Packet 3 - This is a duplicate of packet 2, with a 0 delta time
from this point on the conversation is hung.

Linux just keeps sending the same packet, backin of on the time between
retries.

This is reproducible.

Any ideas???

Regards

Steve

> TRIGGER - ,ev2 conva
> Packet 0, Saved length=74, Real length=74, T=23999mS T=23999mS
> Frame arrived at 11:18:17.89
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 60 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dcf (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30517 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Synchronise
> Sequence Number = 2432354582
> Window = 32120
> Checksum = 4294949841
> Maximum segment size = 1460
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 convb
> Packet 1, Saved length=60, Real length=60, T=24001mS T=2mS
> Frame arrived at 11:18:17.91
> From pserv To Linux2 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 44 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = 2c1a (hex)
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 60 hops
> Protocol = 6 == TCP
> Header Checksum = 4294941946 : Checksum is Correct
> Source Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> Destination Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 515 Unknown
> Destination Port = 1022 Unknown
> Control = Ack Synchronise
> Sequence Number = 635539295 - Acknowledgement Number = 2432354583
> Window = 0
> Checksum = 4294954250
> Maximum segment size = 1460
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 003A 00 00 ..
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 2, Saved length=60, Real length=60, T=24001mS T=0mS
> Frame arrived at 11:18:17.91
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dd0 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30536 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
>
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 3, Saved length=60, Real length=60, T=24001mS T=0mS
> Frame arrived at 11:18:17.91
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dd1 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> _Protocol = 6 == TCP
> Header Checksum = 30535 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 convb
> Packet 4, Saved length=60, Real length=60, T=24006mS T=5mS
> Frame arrived at 11:18:17.96
> From pserv To Linux2 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = 2d1a (hex)
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 60 hops
> Protocol = 6 == TCP
> Header Checksum = 4294941694 : Checksum is Correct
> Source Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> Destination Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 515 Unknown
> Destination Port = 1022 Unknown
> Control = Ack
> Sequence Number = 635539296 - Acknowledgement Number = 2432354583
> Window = 4096
> Checksum = 4294956231
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 20 45 44 46 43 44 EDFCD
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 5, Saved length=60, Real length=60, T=24006mS T=0mS
> Frame arrived at 11:18:17.96
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dd2 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30534 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 6, Saved length=60, Real length=60, T=24199mS T=193mS
> Frame arrived at 11:18:17.289
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dd5 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30531 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 7, Saved length=60, Real length=60, T=24599mS T=400mS
> Frame arrived at 11:18:17.689
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dd8 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30528 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 8, Saved length=60, Real length=60, T=25399mS T=800mS
> Frame arrived at 11:18:18.489
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = dd9 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30527 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 9, Saved length=60, Real length=60, T=27000mS T=1601mS
> Frame arrived at 11:18:20.90
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = de4 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30516 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 10, Saved length=60, Real length=60, T=30201mS T=3201mS
> Frame arrived at 11:18:23.291
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = e00 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30488 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> _____________________________________________________________________
> ,ev2 conva
> Packet 11, Saved length=60, Real length=60, T=36604mS T=6403mS
> Frame arrived at 11:18:29.694
> From Linux2 To pserv 0800 == DOD IP
> L2:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> L3:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Internet Protocol
> Datagram Length = 40 octets.
> Version = 04
> IP Precedence - Routine : Normal Delay, Normal Throughput, Low Reliability
> Identifier = e21 (hex)
> Do not fragment...
> Last fragment. - Fragment offset = 0 (bytes)
> Time to live = 64 hops
> Protocol = 6 == TCP
> Header Checksum = 30455 : Checksum is Correct
> Source Address - ?:193.195.25.39 (Net: 193. 195. 25 Host: 39 )
> Destination Address - ?:193.195.25.10 (Net: 193. 195. 25 Host: 10 )
> L4:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> TCP
> Source Port = 1022 Unknown
> Destination Port = 515 Unknown
>
> Control = Ack
> Sequence Number = 2432354583 - Acknowledgement Number = 635539296
> Window = 32120
> Checksum = 26447
> L5:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Session Protocol Unknown
> Offset from Start of Packet
> 0036 00 00 00 00 00 00 ......
> L6:._ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
>
> --
> Steve Hanselman
> Brendata (UK) Ltd
>

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/