Re: _High_ CPU usage while routing (mostly) small UDP packets

[Karsten Desler]

* P@xxxxxxxxxxxxxx wrote:
> Karsten Desler wrote:
> >* David S. Miller wrote:
> >
> >>It's spending nearly half of it's time in iptables.
> >>Try to consolidate your rules if possible.  This is the
> >>part of netfilter that really doesn't scale well at all.
> >>
> >
> >Removing the iptables rules helps reducing the load a little, but the
> >majority of time is still spent somewhere else.
> 
> Well with NAPI it can be hard to tell CPU usage.
> You may need to use something like cyclesoak to get
> a true idea of CPU left.

Thanks, I'll look into it.

> Also have a look at http://www.hipac.org/ as netfilter
> has silly scalability properties.

I did before, but I read on Harald Weltes' weblog that 2.4 gives
slightly worse pps results than 2.6, and since the cpu usage is as high
as it is, I didn't want to take any more performance hits.
I'll try to see what performance impact the netfilter rules have during
peak load.

> I also notice that a lot of time is spent allocating
> and freeing the packet buffers (and possible hidden
> time due to cache misses due to allocating on one
> CPU and freeing on another?).
> How many [RT]xDescriptors do you have configured by the way?

256. I increased them to 1024 shortly after the profiling run, but
didn't notice any change in the cpu usage (will try again with cyclesoak).

> Anyway attached is a small patch that I used to make the e1000
> "own" the packet buffers, and hence it does not alloc/free
> per packet at all. Now this has only been tested in one
> configuration where I was just sniffing the packets, so
> definitely YMMV.

Thanks, I'll give it a spin.

Cheers,
 Karsten
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/