Re: RCU latency regression in 2.6.16-rc1

[Ingo Molnar]

* Lee Revell <rlrevell@xxxxxxxxxxx> wrote:

> On Sat, 2006-01-28 at 20:52 +0100, Eric Dumazet wrote:
> > > Your new trace shows that we are held up in in rt_run_flush(). 
> > > I guess we need to investigate why we spend so much time in rt_run_flush(),
> > > because of a big route table or the lock acquisitions.
> > 
> > Some machines have millions of entries in their route cache.
> > 
> > I suspect we cannot queue all them (or only hash heads as your
> > previous patch) by RCU. Latencies and/or OOM can occur.
> > 
> > What can be done is :
> > 
> > in rt_run_flush(), allocate a new empty hash table, and exchange the
> > hash tables.
> > 
> > Then wait a quiescent/grace RCU period (may be the exact term is not
> > this one, sorry, I'm not RCU expert)
> > 
> > Then free all the entries from the old hash table (direclty of course,
> > no need for RCU grace period), and free the hash table.
> > 
> > As the hash table can be huge, we might need allocate it at boot time,
> > just in case a flush is needed (it usually is :) ). If we choose
> > dynamic allocation and this allocation fails, then fallback to what is
> > done today.
> > 
> 
> No problem, I'm not a networking expert...
> 
> Ingo's response to these traces was that softirq preemption, which 
> simply offloads all softirq processing to softirqd and has been tested 
> in the -rt patchset for over a year, is the easiest solution.  Any 
> thoughts on that?  Personally, I'd rather fix the very few problematic 
> softirqs, than take such a drastic step - this softirq appears to be 
> one of the last obstacles to being able to meet a 1ms soft RT 
> constraint with the mainline kernel.

well, softirq preemption is not really a drastic step - its biggest 
problem is that it cannot be included in v2.6.16 ;-) But i agree that if 
a solution can be found to break up a latency path, that is preferred.

	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/