s2idle resume hangs in ohci/ehci on RK3588: HC registers touched before the USB2 PHY is powered back on

[Igor Paunovic]

Hi,

on an Orange Pi 5 Plus (RK3588, mainline 7.3-rc1 based tree, generic-ohci/generic-ehci with
phy-rockchip-inno-usb2) resume from suspend-to-idle reliably hangs in the USB 2.0 host
controller resume path. The CPU that runs ohci_platform_resume() never returns (RCU stall,
"CPUs still haven't responded to the NMI"), everything that waits on it in dpm_resume()
stalls behind it, and the board needs a cold reset. With the four USB 2.0 host controllers
unbound before suspend the same s2idle cycle completes every time (RTC alarm wake, full
resume, 3 s), so the rest of the platform is fine.

Wake-up itself works: the board is woken by the hym8563 RTC alarm (PM: Triggering wakeup
from IRQ 52) - that needed a separate dts fix which I sent yesterday [1].

To find where it stops I put kprobes (with tp_printk) on the resume path. For the two
OHCI controllers, same kernel, same cycle:

  fc8c0000.usb (host1, survives):
    ohci_platform_resume -> ohci_platform_power_on (clocks only) -> 0
    ohci_resume  entered        @142.838
    usb_hcd_resume_root_hub     @142.859   (the "powerup ports" + msleep(20) branch)
    ohci_resume  returned 0
    ... later, root hub resume: usb_phy_roothub_resume -> rockchip_usb2phy_init/power_on
        -> ohci_rh_resume
  fc840000.usb (host0, hangs):
    ohci_platform_resume -> ohci_platform_power_on (clocks only) -> 0
    ohci_resume  entered        @142.994
    (nothing else, ever)

So the hang is at the first HC register access in ohci_resume() (the ohci_readl() of
HcControl / the port power / intrenable writes), which happens before the USB2 PHY is
powered on again: since the PHY handling moved into the HCD core (usb_phy_roothub), the
PHYs are powered off in hcd_bus_suspend() and powered on in hcd_bus_resume() (both only for
system sleep, !PMSG_IS_AUTO), i.e. at root hub level, and the root hub resumes after its
parent controller. At probe time the order is the opposite: usb_add_hcd() does
usb_phy_roothub_power_on() before hcd->driver->reset(). That matches what I see: after a
"hosts unbound" s2idle cycle, binding the drivers again (probe path) reliably works, while
the resume path hangs.

On RK3588 the inno-usb2 PHY powers down its PLL/refclk/bias blocks while suspended
(phy-rockchip-inno-usb2.c, comment in rockchip_usb2phy_power_on() about common_on_n and the
reset done on power-on), and the OHCI/EHCI controllers take one of their clocks from that
PHY (clocks = <&cru HCLK_HOST0>, ..., <&u2phy2>). So an AHB access to the controller while
its PHY is still suspended has no clock to complete on, and the access never returns - which
would explain the "no reaction to NMI" symptom (this is my best explanation so far, not
confirmed with a bus or clock trace). The Rockchip vendor tree avoids this by giving the PHY
driver system PM ops that reset and re-tune the PHY on resume, before the consumers run
(rockchip-linux/kernel, develop-6.1, phy-rockchip-inno-usb2.c, rockchip_usb2phy_pm_resume():
"PHY lost power in suspend, it needs to reset PHY to recovery clock to usb controller").

Data points (all dvfs test kernel, 7.3.0-rc1 based; every "hang" needed a cold reset):
  - all 4 USB2 hosts bound, s2idle:            hang (ehci x2 + ohci calling, none returned)
  - EHCI unbound, OHCI bound:                  hang in ohci_resume of fc840000 (2/2)
  - all 4 unbound:                             OK (4/4)
  - all bound, cpuidle limited to WFI:         OK (1/1)   <- timing dependent, see below
  - the surviving controller (fc8c0000) took the "HC state retained" branch of
    ohci_resume(); the hanging one (fc840000) did not get past the first register access.

(The OHCI platform devices resume synchronously from the main dpm_resume() thread - power/async
is disabled for them - which is why nothing else in the resume sequence is printed after the
hang; the EHCI ones are async, so in the all-bound case two ehci and one ohci resume were in
flight when everything stopped.)

Why host1 survives and host0 does not (identical PHY port configs) - reading the PHY GRF
status and the clk enable counts right before suspend, in the test configuration (EHCI
unbound, OHCI bound): the u2phy2 port (host0, nothing plugged in) is already in PHY suspend
(GRF status phy_sus set, no line state) and its usb480m clock has the OHCI as its only user;
the u2phy3 port (host1, a HID dongle plugged in) is not suspended and its clock has more than
one user.

So the port without a device is already put into suspend by the PHY driver's host-port state
machine, and its 480 MHz clock has a single user (the OHCI): ohci_platform_suspend() drops it
to zero and the clock output is gated; ohci_platform_resume() re-enables the clock, but the PHY
itself stays suspended (PLL down) until the root hub resume powers it on, so the first HC
register access would have no clock to complete on. The port with a device connected keeps its
PHY awake and its clock never reaches zero, so the same code path survives there. With the EHCI
siblings bound the outcome depends on timing (the async EHCI root hub may power the shared PHY
on before the synchronous OHCI resume touches its registers), which would match the mixed
results. Also: after suspend the OHCI ends up in the RCU-stall/no-NMI-response state, i.e. the
CPU is stuck in the bus access, not in a software wait.

Questions:
  1. Is the intended fix to power the roothub PHYs on before the controller's own resume
     touches the hardware (e.g. in ohci_resume()/ehci_resume(), or a usb_phy_roothub_resume()
     call from the platform glue before ohci_resume()), or should this be handled in the
     Rockchip PHY driver with system PM ops as the vendor tree does?
  2. Has anyone got s2idle + USB2 host working on RK3588 mainline? I could not find a report
     on lore.

I can test patches on this board (UART console logging is in place), or try one myself
for whichever layer you think is right - I have not attempted a fix yet because that choice
is the question. Per Documentation/process/coding-assistants.rst: the kprobe placement and
the log triage above were done with the help of an LLM assistant; all measurements are from
the board and the reproducer is the unbind/bind matrix above.

[1] https://lore.kernel.org/all/20260906181622.11991-1-royalnet026@xxxxxxxxx/

Kernel: 7.3.0-rc1 based (drm-misc-next + accel/rocket DVFS series), BL31 v2.12.0-10-g70d814213
(v2.12.0 plus one local cherry-pick unrelated to USB), Orange Pi 5 Plus.

Igor