Re: [PATCH] nvme-multipath: add fail_io_now sysfs attribute to fail queued I/O

[Krishna Iyer]

On 9/11/26 2:37 PM, Sagi Grimberg wrote:

> At that point you can disconnect the controllers, which will teardown
> the inflight blocked IO.

That works, but it tears down every namespace on the controller and
someone has to reconnect afterwards. Our controllers back multiple
namespaces and we only want to fail I/O for one of them.

> If you have a namespace that is attached to controllers say X,Y,Z,W
> Now all of these controllers are unavailable, aren't all of the namespaces
> on these paths also unavailable?

Yes, they are all pathless. The difference between them is what to do
with the queued I/O. Holding it until a path returns is right for a
namespace whose user is still around, and wrong for one whose user is
gone. In our case a SIGKILLed VMM will never collect its completions,
while the neighboring namespaces belong to running VMs that should
keep queueing and resume once the fabric heals. The kernel cannot tell
these apart, only host software knows. That is why this needs to be a
per-namespace policy set by the admin, the same model as
fail_if_no_path in dm-multipath.

> Or is it that the paths are online, but the ANA group got into a state where
> there is no optimized path?

In our case no, the controllers sit in CONNECTING for the whole
outage, so there is no LIVE path. But v2 will cover that ANA case too:
with the policy set, a LIVE path with ANA inaccessible or
persistent-loss also does not count as usable.

Either way the no-path condition may be controller-wide, but the
remedy is not: anything controller-scoped releases I/O for every
namespace behind it, and we only want to release one. Hence a
per-namespace policy, which per Nilay's review v2 will rename to
fail_if_no_path and make persistent instead of self-clearing.

Thanks,
Krishna