[RFC PATCH v2 0/4] virtio: add noirq system sleep PM callbacks for virtio-mmio

[Sungho Bae]

From: Sungho Bae <baver.bae@xxxxxxx>

Hi all,

Some virtio-mmio based devices, such as virtio-clock or virtio-regulator,
must become operational before other devices have their regular PM restore
callbacks invoked, because those other devices depend on them.

Generally, PM framework provides the three phases (freeze, freeze_late,
freeze_noirq) for the system sleep sequence, and the corresponding resume
phases. But, virtio core only supports the normal freeze/restore phase,
so virtio drivers have no way to participate in the noirq phase, which runs
with IRQs disabled and is guaranteed to run before any normal-phase restore
callbacks.

This series adds the infrastructure and the virtio-mmio transport
wiring so that virtio drivers can implement freeze_noirq/restore_noirq
callbacks.

Design overview
===============

The noirq phase runs with device IRQ handlers disabled and must avoid
sleepable operations. The main constraints addressed are:

 - might_sleep() in virtio_add_status() and virtio_features_ok().
 - virtio_synchronize_cbs() in virtio_reset_device() (IRQs are already
   quiesced).
 - spin_lock_irq() in virtio_config_core_enable() (not safe to call
   with interrupts already disabled).
 - Memory allocation during vq setup (virtqueue_reinit_vring() reuses
   existing buffers instead).

The series provides noirq-safe variants for each of these, plus a new
config_ops->reset_vqs() callback that lets the transport reprogram
queue registers without freeing/reallocating vring memory.

When a driver implements restore_noirq, the device bring-up (reset ->
ACKNOWLEDGE -> DRIVER -> finalize_features -> FEATURES_OK) happens in
the noirq phase. The subsequent normal-phase virtio_device_restore()
detects this and skips the redundant re-initialization.

Patch breakdown
===============

Patch 1 is a preparatory refactoring with no functional change.
Patches 2-3 add the core infrastructure. Patch 4 wires it up for
virtio-mmio.

 1. virtio: separate PM restore and reset_done paths

    Splits virtio_device_restore_priv() into independent
    virtio_device_restore() and virtio_device_reset_done() paths,
    using a shared virtio_device_reinit() helper. This is a pure
    refactoring to make the restore path independently extensible
    without complicating the boolean dispatch.

 2. virtio_ring: export virtqueue_reinit_vring() for noirq restore

    Adds a thin exported wrapper around the existing static
    virtqueue_reset_split()/virtqueue_reset_packed() helpers, which
    reset vring indices and descriptor state in place without any
    memory allocation. This makes them usable from the transport's
    reset_vqs() callback in noirq context.

 3. virtio: add noirq system sleep PM infrastructure

    Adds noirq-safe helpers (virtio_add_status_noirq,
    virtio_features_ok_noirq, virtio_reset_device_noirq,
    virtio_config_core_enable_noirq, virtio_device_ready_noirq) and
    the freeze_noirq/restore_noirq driver callbacks plus the
    config_ops->reset_vqs() transport hook. Modifies
    virtio_device_restore() to skip bring-up when restore_noirq
    already ran.

 4. virtio-mmio: wire up noirq system sleep PM callbacks

    Implements vm_reset_vqs() which iterates existing virtqueues,
    reinitializes the vring state via virtqueue_reinit_vring(), and
    reprograms the MMIO queue registers. Adds
    virtio_mmio_freeze_noirq/virtio_mmio_restore_noirq and registers
    them via SET_NOIRQ_SYSTEM_SLEEP_PM_OPS().

Testing
=======

Build-tested with arm64 cross-compilation.
(make ARCH=arm64 M=drivers/virtio)

Runtime-tested on an internal virtio-mmio platform with virtio-clock,
confirming that the clock device works well before other devices' normal
restore() callbacks run.

Questions for reviewers
=======================

 - virtio_config_core_enable_noirq() is currently a separate helper
   that duplicates the spin_lock logic, but it only differs from
   virtio_config_core_enable() by the locking context.
   Would it be better to merge them into a single function with irqsafe
   locking, or is the current separate-helper approach acceptable?

 - Should reset_vqs be a mandatory callback when restore_noirq is
   used, or should the core silently skip vq re-initialization when
   the vq list is empty (current behavior)?

 - Some functions, such as get_status() and set_status(), are also used
   in the noirq restore path. Therefore, it is assumed that drivers using
   noirq callbacks have implemented these functions usable in the noirq
   context.
   Is the current implementation acceptable? Or is it necessary to create a
   flag to indicate the driver supports them in the noirq context, or
   have a separate noirq callback?

Changes
=======

v2:
  virtio-mmio: wire up noirq system sleep PM callbacks
   - The code that was duplicated in vm_setup_vq() and vm_reset_vqs() has
     been moved to vm_active_vq() function.


Sungho Bae (4):
  virtio: separate PM restore and reset_done paths
  virtio_ring: export virtqueue_reinit_vring() for noirq restore
  virtio: add noirq system sleep PM infrastructure
  virtio-mmio: wire up noirq system sleep PM callbacks

 drivers/virtio/virtio.c       | 234 ++++++++++++++++++++++++++++++----
 drivers/virtio/virtio_mmio.c  | 131 +++++++++++++------
 drivers/virtio/virtio_ring.c  |  19 +++
 include/linux/virtio.h        |   8 ++
 include/linux/virtio_config.h |  29 +++++
 include/linux/virtio_ring.h   |   3 +
 6 files changed, 359 insertions(+), 65 deletions(-)

-- 
2.43.0