[PATCH RFC 7/8] NFS: Reduce nfsiod workqueue contention

[Chuck Lever]

The default affinity scope for unbound workqueues is now
WQ_AFFN_CACHE_SHARD, which splits each LLC into shards of about
eight cores. On a single-socket system whose LLC fits in one
shard, every NFS I/O completion serializes on one nfsiod pool
lock. Profiling 4KB random writes over NFSv3/RDMA with nconnect=3
shows that lock consuming 17% of CPU cycles: 8% dequeuing work and
9% enqueuing follow-on work from rpciod and nfsiod workers.

Set nfsiod's affinity scope to WQ_AFFN_SMT so each CPU gets its
own pool and queue_work_on() no longer takes a lock on another
CPU. Enqueue contention disappears and dequeue contention drops
to 1.4%. Throughput is unchanged because the workload is
transport-limited, but the freed cycles cut submission latency
variance by 67% (slat stdev 31.6 us to 10.5 us), IOPS stdev by
31%, and p99.9 completion latency by 11%.

Suggested-by: Tejun Heo <tj@xxxxxxxxxx>
Signed-off-by: Chuck Lever <cel@xxxxxxxxxx>
---
 fs/nfs/inode.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 107a2135029d..8e0d2bebba54 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -2618,11 +2618,26 @@ static void nfsiod_stop(void)
  */
 static int nfsiod_start(void)
 {
+	struct workqueue_attrs *attrs;
+
 	dprintk("RPC:       creating workqueue nfsiod\n");
 	nfsiod_workqueue = alloc_workqueue("nfsiod",
 				WQ_MEM_RECLAIM | WQ_UNBOUND | WQ_SYSFS, 0);
 	if (nfsiod_workqueue == NULL)
 		return -ENOMEM;
+	attrs = alloc_workqueue_attrs();
+	if (attrs) {
+		int err;
+
+		attrs->affn_scope = WQ_AFFN_SMT;
+		err = apply_workqueue_attrs(nfsiod_workqueue, attrs);
+		free_workqueue_attrs(attrs);
+		if (err)
+			pr_warn("nfsiod: failed to set SMT affinity scope: %d\n",
+				err);
+	} else {
+		pr_warn("nfsiod: failed to allocate workqueue attrs\n");
+	}
 #if IS_ENABLED(CONFIG_NFS_LOCALIO)
 	/*
 	 * localio writes need to use a normal (non-memreclaim) workqueue.

-- 
2.55.0