[PATCH RFC 6/8] SUNRPC: Reduce rpciod workqueue contention

[Chuck Lever]

rpciod drives the RPC client state machine. Under heavy NFS
workloads, multiple CPUs queue RPC task completions concurrently and
contend on the UNBOUND worker pool lock. perf profiles on a 12-core
system show 30-40% of cycles lost to
native_queued_spin_lock_slowpath in the rpciod pool at the
WQ_AFFN_CACHE scope (one pool per LLC). The WQ_AFFN_CACHE_SHARD
default helps little here, because its 8-core shards split this
system into just two pools of six cores each.

Set WQ_AFFN_SMT on rpciod so each SMT group gets its own pool and
lock. Most UNBOUND workqueues never contend on the pool lock and
profit from a coarser scope's cache locality. rpciod's sustained
completion traffic makes the lock a first-order bottleneck, so the
override belongs on this workqueue rather than in the system-wide
default. Idle kworkers are culled, so the extra pools cost little on
large systems.

Suggested-by: Tejun Heo <tj@xxxxxxxxxx>
Signed-off-by: Chuck Lever <cel@xxxxxxxxxx>
---
 net/sunrpc/sched.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c
index c31cf55b933f..b84af2c0b104 100644
--- a/net/sunrpc/sched.c
+++ b/net/sunrpc/sched.c
@@ -1266,6 +1266,25 @@ void rpciod_down(void)
 	module_put(THIS_MODULE);
 }
 
+static void rpc_set_wq_smt_affinity(struct workqueue_struct *wq,
+				    const char *name)
+{
+	struct workqueue_attrs *attrs;
+	int err;
+
+	attrs = alloc_workqueue_attrs();
+	if (!attrs) {
+		pr_warn("%s: failed to allocate workqueue attrs\n", name);
+		return;
+	}
+	attrs->affn_scope = WQ_AFFN_SMT;
+	err = apply_workqueue_attrs(wq, attrs);
+	free_workqueue_attrs(attrs);
+	if (err)
+		pr_warn("%s: failed to set SMT affinity scope: %d\n",
+			name, err);
+}
+
 /*
  * Start up the rpciod workqueue.
  */
@@ -1280,6 +1299,7 @@ static int rpciod_start(void)
 	wq = alloc_workqueue("rpciod", wq_flags, 0);
 	if (!wq)
 		goto out_failed;
+	rpc_set_wq_smt_affinity(wq, "rpciod");
 	rpciod_workqueue = wq;
 	wq = alloc_workqueue("xprtiod", wq_flags, 0);
 	if (!wq)

-- 
2.55.0