Re: [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock

From: Ivan Vecera

Date: Wed Nov 18 2026 - 10:30:00 EST


On 11/17/26 03:16, Jakub Kicinski wrote:
On Wed, 11 Nov 2026 17:40:11 +0100 Ivan Vecera wrote:
dpll_lock protects topology and object lifetime, but it also serializes
all driver callbacks in the system, so one slow device stalls every other
DPLL device, whichever driver it belongs to. mlx5 callbacks issue firmware
commands with a 60 s default timeout, and zl3073x callbacks can wait
seconds on a driver lock held across ToD writes and phase steps.

Can you provide some real life data of how much this actually helps?
Real device topology + scenario you hit?

The system is GNR-D with a dual complex E825 and a ZL30732 on the
board. The zl3073x has its own PHC, takes the recovered clocks of the
E825 ports and feeds the TX and time reference clocks of the E825. Each
ice PF connects to the zl3073x like this:

+------------------------- zl3073x --------------------------+
| EEC DPLL, PPS DPLL |
| |
| input pin output pin |
| (recovered clock of E825) 156M25_NAC0_CLKREF_SYNC |
+-------^---------------------------------|------------------+
| parent pin | registered with
| dpll_pin_on_pin_register() | dpll_pin_register()
+-------|---------------------------------v------------------+
| RCLK pin TXC DPLL |
+------------------------- ice PF ---------------------------+

The PF owning the source timer also registers the zl3073x output pin
156M25_NAC0_TIMEREF_SYNC with its TSPLL DPLL.

A step of the zl3073x PHC, which ptp4l and ts2phc do at start and after
a large offset, waits for the next 1 PPS edge, up to 1 s, with the
channel lock held. A phase step can hold it for up to 3 s while it
waits for the previous one. A dpll callback of that channel waits for
the lock under dpll_lock, and every other DPLL request and notification
in the system waits behind it. With ToD writes running, requests to an
ice DPLL took 1448 ms in median and 3.9 s at most. With the series they
took 0.2 ms and 0.3 ms, the same as idle.

The bigger reason are zl3073x callbacks we are working on, which have
to wait for the hardware themselves:

- a device reset with resync waits for the fast lock, up to 1 s with a
1 PPS input
- a glitchless stop or start of an output completes on an output edge,
so it takes up to one period, 1 s for a 1 PPS output

Holding the global dpll_lock for that long would stall every DPLL
device in the system. For the output state we tried an asynchronous
implementation, but then the callback returns success before it knows
the result. A failure can only be reported by a later notification, and
the next request gets -EBUSY until the change completes. With a lock
per chip the callback can simply wait, and one chip cannot do these
operations in parallel anyway.

For the cross-domain pins - shouldn't we stretch the domain to cover
all interconnected objects on the system instead of hoping people get
lock ordering right? You documented some rules in the doc, admittedly,
but not sure how enforceable and practical they are.

Stretching the domains would not improve anything here. The only
in-tree registrations across domains are the ones in the diagram above,
and every PF connects to the zl3073x, so the zl3073x and all ice PFs
would end up in one domain. The system above would then stall the same
way as with the global lock.

There is no lock ordering to get right. Only one domain lock is ever
held and all of them share one lockdep class, so any nesting is
reported. __dpll_{device,pin}_change_ntf() assert the domain lock of
the object, so a callback notifying an object of another domain is
reported as well. The rule that a callback must not call functions
taking dpll_lock holds on main already. What lockdep cannot check is
that a driver registering a pin of another driver protects the state
these callbacks share with its own ones. ice is the only such driver
and uses pf->dplls.lock for it.

Thanks,
Ivan