[PATCH] nbd: fix race between nbd_pending_cmd_work and socket teardown

[Weisson]

Hi,

This patch fixes a NULL pointer dereference in nbd_pending_cmd_work()
that occurs when socket teardown races with the partial-send worker.

The bug was reported by Peiyang He via syzkaller fuzzing and exists
since commit 8337b029f788 ("nbd: fix partial sending") introduced the
partial-send worker mechanism.

The race sequence:

  CPU 0 (submit path)              CPU 1 (disconnect path)
  â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??            â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??
  nbd_send_cmd() interrupted
    was_interrupted() && sent > 0
    nbd_sched_pending_work():
      nsock->pending = req
      schedule_work(&nsock->work)
    return BLK_STS_OK
                                   nbd-client -d
                                     sock_shutdown()
    â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??         mutex_lock(&nsock->tx_lock)
    â?? async gap: work is   â??         nbd_mark_nsock_dead():
    â?? queued but kworker   â??           nsock->dead = true
    â?? hasn't run yet       â??           nsock->pending = NULL  â?? cleared
    â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??â??         mutex_unlock(&nsock->tx_lock)

  kworker scheduled
    nbd_pending_cmd_work():
      req = nsock->pending         â?? NULL
      blk_mq_rq_to_pdu(NULL)      â?? accesses 0 + 0xf8
      *** NULL pointer dereference ***

The root cause is that schedule_work() only enqueues the work item;
actual execution depends on kworker scheduling.  Between enqueue and
execution, disconnect can synchronously clear nsock->pending.

The fix establishes single ownership: once nbd_sched_pending_work()
schedules the worker, only the worker may clear nsock->pending and
terminate the request.  Socket teardown (nbd_mark_nsock_dead) only
sets nsock->dead without touching the pending request.  The worker
checks nsock->dead after each send attempt and completes the request
with BLK_STS_IOERR.

Reproduction:

The natural race window is microseconds wide, so a kprobe is used to
inject a busy-wait at nbd_pending_cmd_work() entry to widen it.

1. Build the kprobe delay module (must use mdelay, not msleep --
   kprobe pre-handlers run with preemption disabled):

  /* nbd-delay-repro.c */
  #include <linux/module.h>
  #include <linux/kprobes.h>
  #include <linux/delay.h>

  static int delay_ms = 30;
  module_param(delay_ms, int, 0644);
  static struct kprobe kp;

  static int __kprobes handler_pre(struct kprobe *p, struct pt_regs *regs)
  {
      if (delay_ms > 0)
          mdelay(delay_ms);
      return 0;
  }

  static int __init init(void)
  {
      kp.symbol_name = "nbd_pending_cmd_work";
      kp.pre_handler = handler_pre;
      return register_kprobe(&kp);
  }

  static void __exit exit(void)
  {
      unregister_kprobe(&kp);
  }

  module_init(init);
  module_exit(exit);
  MODULE_LICENSE("GPL");

  Build with:
    make -C /lib/modules/$(uname -r)/build M=$PWD modules

2. Setup environment (forces partial send by filling TCP buffer):

  modprobe nbd
  insmod nbd-delay-repro.ko delay_ms=30
  sysctl -w kernel.panic_on_oops=0
  sysctl -w net.ipv4.tcp_wmem='1024 2048 4096'
  tc qdisc add dev lo root netem delay 20ms
  dd if=/dev/zero of=/tmp/nbd.img bs=1M count=512 status=none
  nbd-server 10823 /tmp/nbd.img &
  sleep 2

3. Trigger (repeat until crash, typically 1-5 iterations):

  nbd-client 127.0.0.1 10823 /dev/nbd1
  echo none > /sys/block/nbd1/queue/scheduler
  # Writer: 1MB O_DIRECT writes with SIGALRM every 500us
  /tmp/nbd-repro-writer 1048576 500 /dev/nbd1 &
  WP=$!
  sleep 0.05
  dmesg -C
  nbd-client -d /dev/nbd1
  sleep 0.2
  kill -9 $WP 2>/dev/null; wait $WP 2>/dev/null
  dmesg | grep 'null pointer'

  The writer source (compile with gcc -O2 -o /tmp/nbd-repro-writer):

  #define _GNU_SOURCE
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <unistd.h>
  #include <signal.h>
  #include <fcntl.h>
  #include <errno.h>
  void h(int s) {}
  int main(int ac, char **av) {
      int bs = ac>1 ? atoi(av[1]) : 1048576;
      int us = ac>2 ? atoi(av[2]) : 500;
      char *buf; posix_memalign((void**)&buf, 4096, bs);
      memset(buf, 0xab, bs); signal(SIGALRM, h);
      int fd = -1;
      while (1) {
          if (fd<0) { fd=open(av[3]?av[3]:"/dev/nbd1",
                              O_WRONLY|O_DIRECT);
                      if (fd<0) { usleep(100000); continue; } }
          ualarm(us, 0);
          ssize_t n = write(fd, buf, bs);
          if (n<0 && errno!=EINTR && errno!=EAGAIN)
              { close(fd); fd=-1; }
      }
  }

  Key parameters explained:
  - "none" scheduler: ensures nbd_queue_rq runs in process context
    via __blk_mq_issue_directly, making SIGALRM visible to
    sk_stream_wait_memory (kworker threads mask all signals)
  - tcp_wmem='1024 2048 4096': tiny send buffer fills immediately
  - netem delay 20ms: delays ACKs, keeps buffer full
  - mdelay(30) kprobe: widens the async gap to 30ms so disconnect
    reliably clears nsock->pending before the worker reads it

4. Expected oops (without fix):

  BUG: kernel NULL pointer dereference, address: 00000000000000f8
  Oops: Oops: 0000 [#1] SMP NOPTI
  Workqueue: events nbd_pending_cmd_work [nbd]
  RIP: 0010:nbd_pending_cmd_work+0x22/0x110 [nbd]
  CR2: 00000000000000f8

5. With fix applied: 300 iterations, zero crashes, zero
   lockdep/WARNING/scheduling-while-atomic reports.

Please review.

Thanks,
Weisson