vfio/pci: guest error recovery proposal

[Michael S. Tsirkin]

>  1) We need to do the right thing for the guest, I don't think we
>     should be presuming that different reset types are equivalent,
>     leaving gaps where we expect the guest/host to do a reset and don't
>     follow through on other reset requests, and we need to notify the
>     guest immediately for the error.
c>  2) We need to do the right thing for the host, that means we should
>     not give the user the opportunity to leave a device in a state
>     where we haven't at least performed a bus reset on link error (this
>     may be our current state and if so we should fix it).

Ok so here is a concrete proposal for improving guest device error
recovery (1).  This is not trying to fix current bugs for 2, but
also does not lock us into not fixing them.

I'll write up proposal for (2) but I feel we can't properly
fix host without fixing (1) first and without breaking compatibility.

Background:

non-fatal errors:

- These errors are due to data link problems.
  The problem is that a transaction was lost, so driver and device are
  out of sync. Device reset is in theory enough to recover from these,
  in practice some drivers might try to do link level reset instead.


fatal errors:

- These errors are due to physical problems.
  The problem is that a transaction was lost, so driver and device are
  out of sync. Link reset might be necessary to recover from these,
  sometimes device reset might be enough for very simple devices.
  If a link above the device reports errors, device might have went away,
  link reset is the only thing that might being it back.

current behaviour:

- vfio will always report that it recovered function from an error.
- whether link reset will trigger depends on whether any other
  function on the same link has a host driver that reports an error.
- also, if there's a host driver that can't handle errors,
  link reset will never trigger


proposed enhancement:

1- allow userspace to request reporting non fatal/fatal errors separately
2- report errors on monitor as events as well
3- forward correct error type to guest
4- set link error flag in userspace (this is optional, used for 5 below)
5- if guest requests link reset, and error flag is set,
  stop vm (I hope we can distinguish this
  from resets that happen on reboot here.
  if yes we might not need error flag in 4 above)


Results:
The advantage of this is that we don't need to manage any state at all.
Most drivers will handle non fatal errors by FLR and will recover fine.
Drivers that attempt link reset will get vmstop which is not
worse than what we have now.

I don't see how this can break any reasonable configuration
that is not already broken, but we might want a flag
to suppress aer reports to guest and just do vmstop
unconditionally.
Alternatively, management can pause vm itself when it sees the error.


Pls remember to Cc qemu list on discussion, not just kvm.

-- 
MST