[QUESTION] xfs: Can we remove flush_workqueue from xlog_cil_push_now?

[Ran Hongyun]

In xlog_cil_push_now(), we call flush_workqueue(cil->xc_push_wq) before
queuing a new work item in sync mode. The intention is to start any
pending background push and reduce the wait time for the new sync work.
However, after commit 636b927eba5b ("workqueue: Make unbound workqueues
to use per-cpu pool_workqueues"), flush_workqueue() iterates over all
nr_cpus pool_workqueue entries, instead of per-NUMA nodes. On large SMP
systems (e.g. 128 CPUs, 4 NUMA nodes), this adds significant overhead.
Even after 85f0d8e39aff ("workqueue: Reduce expensive locks for unbound
workqueue") reduced lock cost, the per-CPU iteration remains costly.
This seems contrary to the original performance intent of the flush.

Below is a bpftrace test result on a 128-CPU system with 4 NUMA nodes,
16 threads doing sequential writes with fsync per write.

Function                    remove flush          with flush
                            per-CPU pwqs     per-CPU      per-NUMA
-------------------------  --------------   ----------   -----------
xfs_fsync_flush_log          32-256 us      256-1000 us    32-256 us
flush_workqueue_prep_pwqs      N/A           2-64 us        2-8 us

With flush_workqueue() removed, xfs_fsync_flush_log latency drops
back to the per‑NUMA range (32‑256 us).

The removal relies on the assumption that flush_workqueue is just a
performance optimization, not a correctness requirement. Without it,
xlog_cil_force_seq already provides a reliable completion guarantee via
xlog_wait(&cil->xc_commit_wait), which blocks until the target sequence
has been committed. The caller will eventually observe the completion
regardless of whether flush_workqueue was called. In addition, ordering
is enforced by xlog_cil_order_write, which ensures all records are
written in strict sequence order. Therefore, removing flush_workqueue
does not weaken any ordering or completion semantics.

If this analysis is correct, the change can be applied broadly to all
sync force callers through xlog_cil_push_now(), as they all share the
same completion and ordering guarantees provided by the existing CIL
infrastructure. The per-CPU iteration overhead would be eliminated on
every sync log force without affecting correctness.

But there are several concerns I'd like to confirm:

Is flush_workqueue just about performance, or does it hide any
correctness dependency I'm missing?

Without it, the "if (!ctx->commit_lsn) { goto restart; }" loop in
xlog_cil_force_seq may be triggered more often. Is that acceptable,
or is it a concern?

Are there any ordering guarantees provided by flush_workqueue that
xc_push_lock + waitqueues don't already provide?

I'd appreciate any thoughts on whether this change is safe.

Thanks,
Ran Hongyun