Re: IO queuing and complete affinity with threads (was Re: [PATCH 0/8] IO queuing and complete affinity)

[Jens Axboe]

On Fri, Feb 08 2008, Nick Piggin wrote:
> On Thu, Feb 07, 2008 at 07:25:45PM +0100, Jens Axboe wrote:
> > Hi,
> > 
> > Here's a variant using kernel threads only, the nasty arch bits are then
> > not needed. Works for me, no performance testing (that's a hint for Alan
> > to try and queue up some testing for this variant as well :-)
> 
> Well this stuff looks pretty nice (although I'm not sure whether the
> softirq->thread changes are a good idea for performance, I guess we'll
> see).

Yeah, that is indeed an open question and why I have two seperate
patches for now (io-cpu-affinity branch and io-cpu-affinity-kthread
branch). As Ingo mentioned, this is how softirqs are handled in the -rt
branch already.

> You still don't have the option that the Intel patch gave, that is,
> to submit on the completer. I guess that you could do it somewhat
> generically by having a cpuid in the request queue, and update that
> with the completing cpu.

Not sure what you mean, if setting queue_affinity doesn't accomplish it.
If you know the completing CPU to begin with, surely you can just set
the queuing affinity appropriately?

> At least they reported it to be the most efficient scheme in their
> testing, and Dave thought that migrating completions out to submitters
> might be a bottleneck in some cases.

More so than migrating submitters to completers? The advantage of only
movign submitters is that you get rid of the completion locking. Apart
from that, the cost should be the same, especially for the thread based
solution.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/