Re: Integration of SCST in the mainstream Linux kernel

[Linus Torvalds]

On Mon, 4 Feb 2008, James Bottomley wrote:
> 
> The way a user space solution should work is to schedule mmapped I/O
> from the backing store and then send this mmapped region off for target
> I/O.

mmap'ing may avoid the copy, but the overhead of a mmap operation is 
quite often much *bigger* than the overhead of a copy operation.

Please do not advocate the use of mmap() as a way to avoid memory copies. 
It's not realistic. Even if you can do it with a single "mmap()" system 
call (which is not at all a given, considering that block devices can 
easily be much larger than the available virtual memory space), the fact 
is that page table games along with the fault (and even just TLB miss) 
overhead is easily more than the cost of copying a page in a nice 
streaming manner.

Yes, memory is "slow", but dammit, so is mmap().

> You also have to pull tricks with the mmap region in the case of writes 
> to prevent useless data being read in from the backing store.  However, 
> none of this involves data copies.

"data copies" is irrelevant. The only thing that matters is performance. 
And if avoiding data copies is more costly (or even of a similar cost) 
than the copies themselves would have been, there is absolutely no upside, 
and only downsides due to extra complexity.

If you want good performance for a service like this, you really generally 
*do* need to in kernel space. You can play games in user space, but you're 
fooling yourself if you think you can do as well as doing it in the 
kernel. And you're *definitely* fooling yourself if you think mmap() 
solves performance issues. "Zero-copy" does not equate to "fast". Memory 
speeds may be slower that core CPU speeds, but not infinitely so!

(That said: there *are* alternatives to mmap, like "splice()", that really 
do potentially solve some issues without the page table and TLB overheads. 
But while splice() avoids the costs of paging, I strongly suspect it would 
still have easily measurable latency issues. Switching between user and 
kernel space multiple times is definitely not going to be free, although 
it's probably not a huge issue if you have big enough requests).

		Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/