Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

[Christoph Hellwig]

On Thu, Oct 06, 2016 at 11:04:15AM +0200, Wouter Verhelst wrote:
> In the current situation, a client could opportunistically send a number
> of write requests immediately followed by a flush and hope for the best.
> However, in that case there is no guarantee that for the write requests
> that the client actually cares about to have hit the disk, a reply
> arrives on the client side before the flush reply arrives. If that
> doesn't happen, that would then mean the client would have to issue
> another flush request, probably at a performance hit.

There is also no guarantee that the server would receive them in order.

Note that people looked into schemes like this multiple times using
a SCSI feature called ordered tags which should provide this sort
of ordering, but no one managed to make it work reliably.

> As I understand Christoph's explanations, currently the Linux kernel
> *doesn't* issue flush requests unless and until the necessary writes
> have already completed (i.e., the reply has been received and processed
> on the client side). Given that, given the issue in the previous
> paragraph, and given the uncertainty introduced with multiple
> connections, I think it is reasonable to say that a client should just
> not assume a flush touches anything except for the writes for which it
> has already received a reply by the time the flush request is sent out.

Exactly.  That's the wording in other protocol specifications, and the
semantics Linux (and Windows) rely on.

> Christoph: just to double-check: would such semantics be incompatible
> with the semantics that the Linux kernel expects of block devices? If
> so, we'll have to review. Otherwise, I think we should go with that.

No, they match the cache flush semantics in every other storage protocol
known to me, and they match the expectations of both the Linux kernel
and any other OS or comsumer I know about perfectly.