[PATCH] sunrpc: fix lost wakeups when stopping service threads

[Kal Cutter Conley]

Stopping NFS can leave rpc.nfsd 0 in an uninterruptible wait with
this stack:

  svc_stop_kthreads [sunrpc]
  svc_set_num_threads [sunrpc]
  nfsd_svc [nfsd]
  write_threads [nfsd]
  nfsctl_transaction_write [nfsd]

SP_NEED_VICTIM and SP_VICTIM_REMAINS remain set while all remaining
nfsd threads sleep in svc_recv() and none has RQ_VICTIM set.

The stop request is published with unordered set_bit() operations.
svc_pool_wake_idle_thread() checks the idle list and task state before
calling wake_up_process(). If it sees an empty list or TASK_RUNNING,
it skips the wakeup and its memory barrier. The requester can therefore
miss a sleeping worker while that worker misses the stop request.

Order work publication before both lockless checks, even when no
wakeup occurs. Pair with set_current_state() and llist_add() in
svc_thread_wait_for_work(), so the worker observes the pending work or
the requester observes the idle worker and wakes it.

Fixes: 60749cbe3d8a ("sunrpc: change sp_nrthreads from atomic_t to unsigned int.")
Signed-off-by: Kal Cutter Conley <kalcutterc@xxxxxxxxxx>
---
 net/sunrpc/svc.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index af8d87c1de83..f3cc63acc5e1 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -29,6 +29,8 @@
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/bc_xprt.h>
 
+#include <asm/barrier.h>
+
 #include <trace/events/sunrpc.h>
 
 #include "fail.h"
@@ -693,6 +695,13 @@ void svc_pool_wake_idle_thread(struct svc_pool *pool)
 	struct svc_rqst	*rqstp;
 	struct llist_node *ln;
 
+	/*
+	 * Order work publication before the idle-list and task-state checks,
+	 * even when wake_up_process() is skipped. Pairs with the barriers in
+	 * set_current_state() and llist_add() in svc_thread_wait_for_work().
+	 */
+	smp_mb();
+
 	rcu_read_lock();
 	ln = READ_ONCE(pool->sp_idle_threads.first);
 	if (ln) {

base-commit: b803428098f6fcd584db9f0499217b80e465bb34
-- 
2.53.0