jgarzik@pobox.com said:
> Further, for NAPI and networking in general, it is recommended to bind
> each NIC to a single interrupt, and never change that binding.
I assume you mean "bind each NIC interrupt to a single CPU" here. I've
done quite a lot of benchmarking on dual SMP that shows that for
high-load networking, you basically have two cases:
- the irq load is less than what can be handled by one CPU. This is the
case, for example, using a NAPI e1000 driver under any load on a
> 1 GHz SMP machine. even with two e1000 cards under extreme load,
one CPU can run the interrupt handlers with cycles to spare (thanks
to NAPI). This config (all NIC interrupts on CPU0) is optimal as
long as CPU doesn't become saturated. Trying to distribute the
interrupt load across multiple CPUs incurs measurable performance
loss, probably due to cache effects.
- the irq load is enough to livelock one CPU. It's easy for this to
happen with gigE NICs on a non-NAPI kernel, for example. In this
case, you're better off binding each heavy interrupt source to a
different CPU.
2.4's default behavior isn't optimal in either case.
> Delivering a single NIC's interrupts to multiple CPUs leads to a
> noticeable performance loss. This is why some people complain that
> their specific network setups are faster on a uniprocessor kernel than
> an SMP kernel.
This is what I've seen as well. The good news is that you can pretty
much recapture the uniprocessor performance by binding all heavy
interrupt sources to one CPU, as long as that CPU can handle it. And any
modern machine with a NAPI kernel _can_ handle any realistic gigE load.
I should mention that these results are all measurements of gigabit
bridge performance, where every frame needs to be received on one NIC
and sent on the other. So there are obvious cache benefits to doing it
all on one CPU.
-- Jason Lunz Reflex Security lunz@reflexsecurity.com http://www.reflexsecurity.com/- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Fri Mar 07 2003 - 22:00:28 EST