[RFC PATCH v2 0/1] drm/amdgpu: MMIO TLB invalidation fallback when KIQ is wedged after S4 resume

[Denis Pisarev]

Hi,

v2 of this RFC, addressing the automated sashiko-bot review of v1.
The failure data and trigger isolation are unchanged from v1
(bugzilla 219492):

- S4 resume on Cezanne (gmc_v9, GFXOFF) -> KIQ TLB flush reg failures
  at 80-140/hour for 9+ hours, sched.ready true throughout; each
  failure burns the ~5 s retry window and the invalidation is dropped
- GFXOFF held off across S4: 0 errors (vs ~70-140/30 min stock);
  re-enabled 40 min after a clean resume: 0 errors in 15 min
- so the wedge forms in the S4 resume window while GFXOFF is allowed

Changes since v1 (all from the bot review, all reasonable):

1. [Critical] the runtime MMIO fallback now holds the GC block awake
   with amdgpu_gfx_off_ctrl(adev, false) across the direct register
   access - the unguarded v1 could touch power-gated registers.
   Because that call may sleep, the fallback is restricted to process
   context.
2. [High] failure counter moved from a global in amdgpu_gmc to the
   per-instance struct amdgpu_kiq - no cross-talk between XCCs.
3. [High] SR-IOV VFs never take the runtime MMIO fallback (they lack
   privileges for direct GMC invalidation register writes).
4. [Medium] gmc_v9_0_flush_gpu_tlb_mmio() (extracted helper) takes
   invalidate_lock with irqsave since the path is now
   runtime-reachable.
5. [High, pre-existing] the MES branch now propagates
   amdgpu_mes_reg_write_reg_wait()'s error instead of hardcoding
   success.
6. [Medium] threshold warning no longer claims a fallback that
   gmc_v10/v11/v12 do not implement.

Open question kept from v1: a KIQ command that timed out remains
queued in the ring; if the ring recovers late, a duplicate
(idempotent) invalidation could race a CPU MMIO flush. Input on
whether the req/ack handshake needs protection for that case is
welcome. Alternative direction also still open: fix the S4 resume
ordering itself (RLC/ME vs GFXOFF) instead of a runtime fallback -
happy to run tracing on the affected hardware.

Patch 1/1 follows.

Denis Pisarev

Denis Pisarev (1):
  drm/amdgpu: fall back to MMIO TLB invalidation when KIQ is
    unresponsive

 drivers/gpu/drm/amd/amdgpu/amdgpu.h     |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c |  18 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h |   2 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c   | 111 ++++++++++++++++++------
 5 files changed, 102 insertions(+), 33 deletions(-)

-- 
2.55.0