[PATCH V0 00/21] accel/amdxdna: Kernel submission and PM for AIE4

[David Zhang]

This patch series extends the amdxdna accelerator driver to support
AMD AIE4 (NPU3) platforms with kernel-mode command submission (KMQ),
device telemetry queries, comprehensive system and runtime power
management, and NPU3 classic device support.

Previously, AIE4 devices relied on userspace doorbell mapping for command
dispatch. This series establishes kernel-mode submission as the default
execution path, providing robust fence management, BO reservation
locking, and queue lifecycle handling. Additionally, it implements full
system suspend/resume and runtime power management across Physical
Functions (PF), Virtual Functions (VF), and Classic devices, along with
SR-IOV PM dependency linking and userspace runtime compatibility stubs.

Series structure:

1. Firmware Interfaces, Versioning & Compatibility (Patches 1-4)
   - Patch 1: Rename NPU3 firmware filenames to match official release
     binaries (npu.sbin, cert.sbin) and declare MODULE_FIRMWARE().
   - Patch 2: Remove userspace doorbell mmap and default to kernel
     submission with invalid doorbell offsets for user contexts.
   - Patch 3: Add CERT (Column Enternal Run Time) firmware version
     queries and protocol validation (aie_check_cert_protocol) before
     upgrading queue layouts.
   - Patch 4: Upgrade firmware interface version to 6.0, updating host
     queue layout, opcode definitions, and hwctx creation requests.

2. Device Operations, Telemetry & Power Modes (Patches 5-9)
   - Patch 5: Add NPU3 classic device operations (aie4_classic_ops) and
     register definitions, managing SMU/PSP firmware symmetrically, and
     remove unreleased device IDs 0x1B0B and 0x1B0C.
   - Patch 6: Expose AIE and NPU firmware versions via GET_INFO ioctl.
   - Patch 7: Add power mode get/set support via GET_INFO and SET_STATE.
   - Patch 8: Restore cached user power mode overrides on hardware start
     so overrides persist across PM cycles and firmware reloads.
   - Patch 9: Add clock metadata, DPM frequency table initialization,
     and hardware resource info queries, generalizing DPM tracking
     across AIE generations in struct aie_device, with independent
     clock domain level handling for AIE compute and host clocks.

3. Hardware Initialization & Transport Hooks (Patches 10-12)
   - Patch 10: Add context switch hysteresis configuration with debugfs
     control (ctx_switch_hysteresis_us) applied on hardware start.
   - Patch 11: Refactor AIE4 hardware initialization sequence into
     distinct modular phases (query_fw, config_fw, setup_aie) across
     PF, VF, and classic devices.
   - Patch 12: Decouple PCI doorbell and MSI-X notification transport
     hooks from transport-neutral context management code.

4. Kernel-Mode Command Submission & Fencing (Patches 13-16)
   - Patch 13: Implement kernel queue lifecycle, job workqueue, and
     memory layout for kernel-mode submission.
   - Patch 14: Fix fence timeline name (dev_name() to prevent UAF after
     context destruction) and allocate unique fence context per job to
     prevent evicting fences in shared BO reservation objects.
   - Patch 15: Prepare command submission structures, smp_rmb ordering
     for read index updates, and asynchronous disconnect/reset checks.
   - Patch 16: Implement full command packet building (direct/indirect
     packets), BO reservation locking and fence attachment matching
     AIE2, queue slot waiting, context reset recovery, and
     aie4_hwctx_wait_for_running() to safely quiesce worker threads.

5. System & Runtime Power Management (Patches 17-20)
   - Patch 17: Fix runtime PM deadlock on device removal by finalizing
     RPM before acquiring dev_lock, and symmetrically initialize RPM
     during probe across all device types.
   - Patch 18: Implement system suspend and resume for PF, VF, and
     classic devices, draining in-flight jobs, restoring contexts, and
     re-linking VFs.
   - Patch 19: Link SR-IOV VFs via device_link_add() to ensure the PM
     core suspends VFs before the PF and resumes the PF before VFs.
   - Patch 20: Implement runtime suspend and resume support for AIE4
     and AIE2, validating active VFs on PF and handling independent VF
     runtime power states.

6. Userspace ABI Compatibility (Patch 21)
   - Patch 21: Add a stub hwctx_config callback returning 0 to support
     the DRM_AMDXDNA_CONFIG_HWCTX ioctl, enabling userspace validation
     runtimes (such as XRT GEMM tests) to run unmodified.

Testing:
- Tested on AMD AIE4/NPU3 hardware in both classic and SR-IOV (PF/VF)
  modes.
- Verified kernel-mode command submission with direct and indirect
  execution packets under concurrent workloads.
- Verified system suspend/resume (S2idle/S3) and runtime autosuspend
  cycles during idle and active command submission.
- Verified SR-IOV VF binding, execution, and PM dependency sequencing.
- Confirmed no regression on existing AIE2 devices (NPU1/NPU4).

David Zhang (21):
  accel/amdxdna: Rename NPU3 firmware files
  accel/amdxdna: Remove mmap for doorbell
  accel/amdxdna: Add CERT firmware version support
  accel/amdxdna: Upgrade firmware version to 6.0
  accel/amdxdna: Add NPU3 classic device support
  accel/amdxdna: Add AIE version query to aie4_get_info
  accel/amdxdna: Add get and set power_mode for AIE4
  accel/amdxdna: Restore power mode override on AIE4 hardware start
  accel/amdxdna: Add clock, DPM frequency, and resource info queries for
    AIE4
  accel/amdxdna: Add context switch hysteresis with debugfs control
  accel/amdxdna: Refactor AIE4 hardware initialization sequence
  accel/amdxdna: Decouple AIE4 doorbell and MSI-X notification transport
    hooks
  accel/amdxdna: Implement AIE4 kernel queue lifecycle and memory layout
  accel/amdxdna: Fix fence timeline name and context allocation
  accel/amdxdna: Prepare for AIE4 command submission
  accel/amdxdna: Implement AIE4 command packet building and submission
  accel/amdxdna: Finalize runtime PM before acquiring dev_lock on
    removal
  accel/amdxdna: Implement AIE4 suspend and resume
  accel/amdxdna: Link SR-IOV VFs for power management sequencing
  accel/amdxdna: Implement runtime suspend and resume support
  accel/amdxdna: Add stub hwctx_config for AIE4

 drivers/accel/amdxdna/aie.c             |   45 +-
 drivers/accel/amdxdna/aie.h             |   45 +-
 drivers/accel/amdxdna/aie2_message.c    |    4 +-
 drivers/accel/amdxdna/aie2_pci.c        |   65 +-
 drivers/accel/amdxdna/aie2_pci.h        |   39 +-
 drivers/accel/amdxdna/aie2_pm.c         |   10 +-
 drivers/accel/amdxdna/aie4_ctx.c        | 1083 +++++++++++++++++++++--
 drivers/accel/amdxdna/aie4_host_queue.h |   79 +-
 drivers/accel/amdxdna/aie4_message.c    |  216 +++++
 drivers/accel/amdxdna/aie4_msg_priv.h   |  137 ++-
 drivers/accel/amdxdna/aie4_pci.c        |  812 ++++++++++++++++-
 drivers/accel/amdxdna/aie4_pci.h        |  138 ++-
 drivers/accel/amdxdna/aie4_sriov.c      |  104 ++-
 drivers/accel/amdxdna/amdxdna_ctx.c     |   29 +-
 drivers/accel/amdxdna/amdxdna_ctx.h     |   22 +
 drivers/accel/amdxdna/amdxdna_debugfs.c |    3 +
 drivers/accel/amdxdna/amdxdna_pci_drv.c |   53 +-
 drivers/accel/amdxdna/amdxdna_pci_drv.h |   18 +-
 drivers/accel/amdxdna/amdxdna_pm.c      |   35 +
 drivers/accel/amdxdna/amdxdna_pm.h      |    4 +-
 drivers/accel/amdxdna/amdxdna_sysfs.c   |    2 +-
 drivers/accel/amdxdna/npu1_regs.c       |   19 +-
 drivers/accel/amdxdna/npu3_regs.c       |   99 ++-
 drivers/accel/amdxdna/npu4_regs.c       |   30 +-
 24 files changed, 2805 insertions(+), 286 deletions(-)

-- 
2.34.1