[PATCH] ceph: unregister aborted requests in ceph_mdsc_wait_request()

[Xiubo Li via B4 Relay]

From: Xiubo Li <xiubo.li@xxxxxxxxx>

When a request is aborted because the waiter was killed or the wait
timed out, the abort path marks it with CEPH_MDS_R_ABORTED but leaves
it parked on a wait list (r_wait) and registered in the request tree.
The waiter's put can then be the last reference while the request is
still linked, freeing it with r_wait non-empty:

  WARNING: fs/ceph/mds_client.c:1233 ceph_mdsc_release_request+0x247/0x250
  WARN_ON_ONCE(!list_empty(&req->r_wait))

The freed request stays behind as a dangling entry on the session's
waiting list, and the next __wake_requests() or kick_requests() walk
uses it after free (a stress test that SIGKILLs waiters during MDS
failovers reproduces this reliably).

Fix the abort path so that the request is removed from its wait list
and from the request tree when the waiter gives up, making the waiter's
put the final reference on a request that is no longer linked anywhere.
Both operations happen under mdsc->mutex; a racing reply or forward may
already have unregistered the request, so only unregister it when it is
still in the tree.

Fixes: e1518c7c0a67a ("ceph: clean up mds reply, error handling")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Xiubo Li <xiubo.li@xxxxxxxxx>
---
The full call trace:

 ------------[ cut here ]------------
 refcount_t: addition on 0; use-after-free.
 WARNING: lib/refcount.c:25 at refcount_warn_saturate+0x6a/0x90, CPU#1: kworker/1:1/99
 Modules linked in: ceph libceph krb5 netfs xsk_diag vsock_diag uinput snd_seq_dummy snd_hrtimer rfkill nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables qrtr sunrpc intel_rapl_msr intel_rapl_common intel_uncore_frequency_common intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_pmc_pwrm_telemetry intel_vsec kvm_intel kvm snd_hda_codec_generic irqbypass snd_hda_intel rapl snd_hda_codec snd_hda_core snd_intel_dspcfg snd_intel_sdw_acpi snd_hwdep snd_seq iTCO_wdt snd_seq_device intel_pmc_bxt snd_pcm i2c_i801 snd_timer pcspkr i2c_smbus snd lpc_ich virtio_balloon soundcore joydev zram lz4hc_compress vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock virtio_net net_failover virtio_gpu failover virtio_dma_buf serio_raw i2c_dev qemu_fw_cfg virtiofs fuse
 CPU: 1 UID: 0 PID: 99 Comm: kworker/1:1 Tainted: G      D             7.2.0-rc7-lockdep+ #13 PREEMPT(lazy)
 Tainted: [D]=DIE
 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.17.0-10.fc44 06/10/2025
 Workqueue: ceph-msgr ceph_con_workfn [libceph]
 RIP: 0010:refcount_warn_saturate+0x6a/0x90
 Code: cc 48 8d 3d b8 f6 d4 01 67 48 0f b9 3a c3 cc cc cc cc 48 8d 3d b7 f6 d4 01 67 48 0f b9 3a c3 cc cc cc cc 48 8d 3d b6 f6 d4 01 <67> 48 0f b9 3a c3 cc cc cc cc 48 8d 3d b5 f6 d4 01 67 48 0f b9 3a
 RSP: 0018:ffffcdb6403a7cc0 EFLAGS: 00010246
 RAX: 0000000000000000 RBX: ffff8d5e6441c000 RCX: ffff8d5e43de6a60
 RDX: ffff8d5e5afbd468 RSI: 0000000000000002 RDI: ffffffffa7fbe040
 RBP: ffffcdb6403a7ce8 R08: 0000000000000001 R09: ffff8d5e5afbd550
 R10: ffffcdb6403a7cc8 R11: 0000000000000000 R12: ffff8d5e6441c000
 R13: ffff8d5e5afbd028 R14: ffff8d5e515c0000 R15: ffff8d5e5a621008
 FS:  0000000000000000(0000) GS:ffff8d61f7904000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 00007f0541dbf000 CR3: 000000011ad8b005 CR4: 0000000000772ef0
 PKRU: 55555554
 Call Trace:
  <TASK>
  __wake_requests+0x3af/0x3d0 [ceph]
  mds_dispatch+0xad/0x240 [ceph]
  ceph_con_process_message+0x88/0x1c0 [libceph]
  ceph_con_v1_try_read+0x316/0x6f0 [libceph]
  ceph_con_workfn+0x1f4/0x500 [libceph]
  ? process_one_work+0x20d/0x600
  process_one_work+0x234/0x600
  worker_thread+0x1e5/0x3c0
  ? __pfx_worker_thread+0x10/0x10
  kthread+0xf5/0x130
  ? __pfx_kthread+0x10/0x10
  ret_from_fork+0x24a/0x360
  ? __pfx_kthread+0x10/0x10
  ret_from_fork_asm+0x1a/0x30
  </TASK>
 irq event stamp: 7923424
 hardirqs last  enabled at (7923423): [<ffffffffa6d5b2a8>] _raw_spin_unlock_irq+0x28/0x50
 hardirqs last disabled at (7923424): [<ffffffffa6d4cdb5>] __schedule+0x685/0x820
---
 fs/ceph/mds_client.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c
index 85f8ceb10377..6b27146d0c9f 100644
--- a/fs/ceph/mds_client.c
+++ b/fs/ceph/mds_client.c
@@ -4024,6 +4024,19 @@ int ceph_mdsc_wait_request(struct ceph_mds_client *mdsc,
 		if (req->r_parent &&
 		    (req->r_op & CEPH_MDS_OP_WRITE))
 			ceph_invalidate_dir_request(req);
+
+		/*
+		 * The waiter is about to drop its reference, which is
+		 * the last one held by the request's caller.  Make sure
+		 * the request is no longer parked on a wait list or
+		 * registered in the request tree, or it could be walked
+		 * (and in the tree case, replied to) after it is freed.
+		 * A racing reply may already have unregistered it, in
+		 * which case it is no longer in the tree.
+		 */
+		list_del_init(&req->r_wait);
+		if (lookup_request(&mdsc->request_tree, req->r_tid) == req)
+			__unregister_request(mdsc, req);
 	} else {
 		err = req->r_err;
 	}

---
base-commit: dee30ce1286a0d18b14545ecac345e4cf4a80511
change-id: 20260827-b4-ceph-fix-abort-request-uaf-new-a98f8c6da00e

Best regards,
--  
Xiubo Li <xiubo.li@xxxxxxxxx>