> > This is heavily demonstrated by ANK's suggested test of bombing a
> > 100Mbit interface with <=128 byte frames, timer interrupts don't even
> > come in. That shows a fundamental problem with how we do interrupts
> > under high load.
>
> No, I really think it's more of a broken small part of the implementation.
>
> So I think it's more a case of
> - we may want to change the way incoming packets are handled to make it
> perform better
> - we may want to throttle excessive incoming interrupts
There are two hardware changes that are addressing this problem:
802.1X Flow Control, which can tell the link partner "shut up for N usec."
Interrupt mitigation hardware on a few new chips.
Triggering flow control requires that the chip run out of receive buffers,
so I've converted all of the PCI drivers over to a "buffer deficit" scheme.
This isn't quite what we want though: flow control is triggered some time
after the driver is refused new receive buffers, not when the interrupt
traffic is high.
Interrupt mitigation is great for overhead, but obviously won't help us with
existing hardware.
> That's a fundamental fact. We can make things go faster, but networking is
> an external entity, and very basic queueing theory applies. If you can't
> keep up, you have to start dropping or it just gets worse. Chapter 1 of
> any queueing theory book. Regardless of how fast you are, if you keep on
> doubling the network speed, it will happen eventually.
Hmmm, I don't see this as a dropping issue. I see as
A simple interrupt flooding problem
A reason to reduce the work done in netif_rx(), so that dropped packets
haven't wasted much work.
A reason putting eth_type_trans() in the drivers was a bad idea (see
previous point)
> Right now I don't see this as a problem in real life. It may become one,
..
> blame). As such, I think people are just making a big thing out of
> nothing.
I pretty much concur. Except that skipping clock ticks is a Bad Thing. I
don't mind dropping packets or characters, but if Xclock is wrong then I'm
later for meetings than I intended to be ;->
> The reason we don't get timer interrupts is not spl-levels. I bet it's
> just a fairly simple issue of CPU starvation. Fixing it may not be simple:
> fairness never is. But let's not panic.
Driver writers assume that timer-chain work may be delayed. But the
real-time clock is sacred.
Donald Becker becker@cesdis.gsfc.nasa.gov
USRA-CESDIS, Center of Excellence in Space Data and Information Sciences.
Code 930.5, Goddard Space Flight Center, Greenbelt, MD. 20771
301-286-0882 http://cesdis.gsfc.nasa.gov/people/becker/whoiam.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/