[PATCH v5 3/5] sunrpc: guarantee a thread per pool when auto-distributing

[Jeff Layton]

svc_set_num_threads() spreads the requested thread count evenly across
the service's pools. In pernode mode each pool maps to a NUMA node, and
svc_pool_for_cpu() steers an incoming transport to the pool for the node
it arrived on. When fewer threads than pools are requested, even
distribution leaves some pools empty, and a transport steered to an
empty pool has no thread to service it.

Floor each pool at one thread when auto-distributing a non-zero count,
so no pool is left empty. Every pool maps to a node that had CPUs when
the pool map was built (svc_pool_map_init_pernode() only creates pools
for nodes returned by for_each_node_with_cpus()), so there is no pool
that should be left threadless. The resulting total may exceed the
requested count. This only affects the auto-distribute path (a
single-value array, i.e. svc_set_num_threads()); callers that set
per-pool counts explicitly via svc_set_pool_threads() are unchanged and
may still set a pool to zero.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@xxxxxxxxxx>
Reviewed-by: NeilBrown <neil@xxxxxxxxxx>
---
 net/sunrpc/svc.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index b942845f82a3..2d1cdf55c561 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -840,6 +840,12 @@ EXPORT_SYMBOL_GPL(svc_set_pool_threads);
  * are multiple pools then the new threads or victims will be distributed
  * evenly among them.
  *
+ * When @nrservs is non-zero but smaller than the number of pools, even
+ * distribution would leave some pools empty. Since each pool maps to a
+ * NUMA node and only services transports steered to that node, every
+ * pool is instead guaranteed at least one thread. The resulting total
+ * may therefore exceed @nrservs.
+ *
  * Caller must ensure mutual exclusion between this and server startup or
  * shutdown.
  *
@@ -855,6 +861,16 @@ svc_set_num_threads(struct svc_serv *serv, unsigned int min_threads,
 	unsigned int remain = nrservs % serv->sv_nrpools;
 	int i, err = 0;
 
+	/*
+	 * Don't let a pool sit empty while threads are being
+	 * auto-distributed: a transport steered to its node would have
+	 * nothing to service it. Every pool maps to a CPU-bearing node,
+	 * so hand each one a thread. This may push the total above
+	 * @nrservs.
+	 */
+	if (base == 0 && nrservs != 0)
+		remain = serv->sv_nrpools;
+
 	for (i = 0; i < serv->sv_nrpools; ++i) {
 		struct svc_pool *pool = &serv->sv_pools[i];
 		int threads = base;

-- 
2.55.0