Re: [PATCH] splice support #2

[Jens Axboe]

On Fri, Mar 31 2006, Ingo Molnar wrote:
> 
> * Jens Axboe <axboe@xxxxxxx> wrote:
> 
> > > > with pipe-based buffering this approach has still the very same problems 
> > > > that sendfile() has with packet boundaries, because it's not enough to 
> > > > have "large enough" buffering (like a pipe has), the pipe also has to be 
> > > > drained, and the networking layer has to know the precise boundary of 
> > > > data.
> > > > 
> > > > the right solution to the packet boundary problem is to pass in a proper 
> > > > "does userspace expect more data right now" flag, or to let userspace 
> > > > 'flush' the socket independently - which is independent of the 
> > > > pipe-in-slice issue. This solution already exists: the MSG_MORE flag.
> > > 
> > > We can add a SPLICE_F_MORE flag for this, right now splice doesn't set
> > > the MSG_MORE flag for the end of the pipe.
> > 
> > Ala
> 
> >  #define SPLICE_F_MOVE	(0x01)	/* move pages instead of copying */
> > +#define SPLICE_F_MORE	(0x02)	/* expect more data */
> 
> ok, nice - something like this should work. The direction of the flag is 
> a philosophical question i guess: i believe in Linux we prefer to 
> default to "buffering enabled", i.e. the default flag should be "expect 
> more data". So maybe it would be better to pass in PLICE_F_END, to 
> indicate end-of-data. [it doesnt mean 'permanent end', so all the files 
> still remain open: this could be something like a HTTP 1.1 pipelined 
> request.]

Hmm I do prefer a MORE as to an END, as it will always give predictable
results. The MORE is a performance hit, where as a missing END wont
necessarily be detected until you start looking at latencies.

> furthermore, the internal implementation should also get smarter and do 
> a flush-socket if it would e.g. block on a pagecache page. [we often 
> prefer a partial packet in such cases instead of having a half-built 
> packet hang around.]

That's a really good idea! Wont be too much trouble to add, probably the
easiest is to add a 'wait' variable to the ->map() function or something
along those lines.

> btw., that 'data boundary' detail is likely lost with the pipe 
> intermediary solution: there is no direct connection between 'input 
> file' and 'output socket', so a 'flush now' event doesnt get propagated 
> in a natural way. (unless we extend pipes with 'data boundary' markers, 
> or force their flushing, which looks a whole lot of complexity for such 
> a simple thing.)

Probably not.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/