Re: [PATCH] RDMA/rxe: Replace use of system_unbound_wq with system_dfl_wq

[yanjun.zhu]

On 3/13/26 8:40 AM, Marco Crivellari wrote:
> This patch continues the effort to refactor workqueue APIs, which has begun
> with the changes introducing new workqueues and a new alloc_workqueue flag:
>
>     commit 128ea9f6ccfb ("workqueue: Add system_percpu_wq and system_dfl_wq")
>     commit 930c2ea566af ("workqueue: Add new WQ_PERCPU flag")
>
> The point of the refactoring is to eventually alter the default behavior of
> workqueues to become unbound by default so that their workload placement is
> optimized by the scheduler.
>
> Before that to happen, workqueue users must be converted to the better named
> new workqueues with no intended behaviour changes:
>
>     system_wq -> system_percpu_wq
>     system_unbound_wq -> system_dfl_wq
>
> This way the old obsolete workqueues (system_wq, system_unbound_wq) can be
> removed in the future.
>
> Link: https://lore.kernel.org/all/20250221112003.1dSuoGyc@xxxxxxxxxxxxx/
> Suggested-by: Tejun Heo <tj@xxxxxxxxxx>
> Signed-off-by: Marco Crivellari <marco.crivellari@xxxxxxxx>

This patch is part of a broader effort to clarify workqueue semantics. 
As discussed in the recent thread at 
https://lore.kernel.org/all/20250221112003.1dSuoGyc@xxxxxxxxxxxxx/], the 
move towards system_dfl_wq is not just a renaming exercise; it's about 
ensuring work items correctly respect the system's housekeeping CPUMASK.

To RXE, it is a software-defined RDMA transport. RXE does not have 
strict hardware-to-CPU affinity requirements. Specifically for the ODP 
prefetch path modified here:

1. Prefetching doesn't rely on being executed on the local CPU where the 
advise_mr was called.

2. The locality benefits of per-cpu execution are negligible compared to 
the importance of system-wide jitter reduction, especially in NOHZ_FULL 
environments.

3. By using system_dfl_wq, we allow the scheduler to offload prefetch 
tasks from isolated CPUs to housekeeping CPUs, which is a desirable 
behavior for real-time users.

The patch is safe, logically sound, and aligns with the current 
kernel-wide modernization of workqueue placement.

I have made tests with this commit. It can work well in functionality.

I am fine with this.

Reviewed-by: Zhu Yanjun <yanjun.zhu@xxxxxxxxx>

Zhu Yanjun

> ---
>   drivers/infiniband/sw/rxe/rxe_odp.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_odp.c b/drivers/infiniband/sw/rxe/rxe_odp.c
> index bc11b1ec59ac..d440c8cbaea5 100644
> --- a/drivers/infiniband/sw/rxe/rxe_odp.c
> +++ b/drivers/infiniband/sw/rxe/rxe_odp.c
> @@ -545,7 +545,7 @@ static int rxe_ib_advise_mr_prefetch(struct ib_pd *ibpd,
>   		work->frags[i].mr = mr;
>   	}
>   
> -	queue_work(system_unbound_wq, &work->work);
> +	queue_work(system_dfl_wq, &work->work);
>   
>   	return 0;
>