Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

[Alex Bligh]

Josef,

> On 15 Sep 2016, at 14:57, Josef Bacik <jbacik@xxxxxx> wrote:
> 
> This isn't an NBD problem, this is an application problem.  The application must wait for all writes it cares about _before_ issuing a flush.  This is the same as for normal storage as it is for NBD.  It is not NBD's responsibility to maintain coherency between multiple requests across connections, just simply to act on and respond to requests.
> 
> I think changing the specification to indicate that this is the case for multiple connections is a good thing, to keep NBD servers from doing weird things like sending different connections to the same export to different backing stores without some sort of synchronization.  It should definitely be explicitly stated somewhere that NBD does not provide any ordering guarantees and that is up to the application.  Thanks,

I don't think that's correct.

The block stack issues a flush to mean (precisely) "do not reply to this until all preceding writes THAT HAVE BEEN REPLIED TO have been persisted to non-volatile storage".

The danger is with multiple connections (where apparently only one flush is sent - let's say down connection 1) that not al the writes that have been replied to on connection 2 have been persisted to non-volatile storage. Only the ones on connection 1 have been persisted (this is assuming the nbd server doesn't 'link' in some way the connections).

There's nothing the 'application' (here meaning the kernel or higher level) can do to mitigate this. Sure it can wait for all the replies, but this doesn't guarantee the writes have been persisted to non-volatile storage, precisely because writes may return prior to this.

-- 
Alex Bligh