[PATCH 0/3] cxl/mbox: support background operation abort requests

From: Davidlohr Bueso
Date: Mon Oct 21 2024 - 23:24:27 EST


Hello,

This series implements the abortable capabilities for mailbox background
operations, introduced in CXL 3.1.

Details in each patch, but patch 1 moves non-sanitize background command
polling out of the mbox_mutex, such that patch 2 can add the machinery
to request aborting the on going command. Patch 3 simply renames a call.

While an overal policy could include command-priorities (which could
be trivially added); the current policy is that, if there is an on going
background operation, a new incoming bg-capable command will blindly
request it to be cancelled, if capable.

Considering the interest of this for Logs[0], perhaps Micron folks could
this? Does this work along the lines of what that discussion concluded?

Applies against Linus' latest tree.

[0]: https://lore.kernel.org/all/20240313071218.729-1-sthanneeru.opensrc@xxxxxxxxxx/

Testing:

lockdep enabled kernel + the qemu counterpart patches:
https://lore.kernel.org/linux-cxl/20240813221255.179200-1-dave@xxxxxxxxxxxx/

#> echo 1 > /sys/bus/cxl/devices/mem1/security/sanitize && sleep 3 && echo 1 > /sys/bus/cxl/devices/mem1/security/sanitize
[ 185.928276] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0d:00.0: Sending command: 0x4400cxl/devices/mem1/security/sanitize
[ 185.930024] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0d:00.0: Doorbell wait took 0ms
[ 185.931608] cxl_pci:__cxl_pci_mbox_send_cmd:306: cxl_pci 0000:0d:00.0: Sanitization operation started
[ 188.936583] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0d:00.0: Sending command: 0x0005
[ 188.943956] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0d:00.0: Doorbell wait took 0ms
[ 188.951786] cxl_pci:cxl_try_to_cancel_background:376: cxl_pci 0000:0d:00.0: Sanitization operation aborted
[ 188.957762] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0d:00.0: Sending command: 0x4400
[ 188.959886] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0d:00.0: Doorbell wait took 0ms
[ 188.962325] cxl_pci:__cxl_pci_mbox_send_cmd:306: cxl_pci 0000:0d:00.0: Sanitization operation started
[ 197.034644] cxl_pci:cxl_mbox_sanitize_work:164: cxl_pci 0000:0d:00.0: Sanitization operation ended

#> cxl update-firmware -F img.fw mem1 && sleep 3 && echo 1 > /sys/bus/cxl/devices/mem1/security/sanitize
[ 14.443484] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0e:00.0: Sending command: 0x0200
[ 14.445884] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0e:00.0: Doorbell wait took 0ms
[ 14.448744] cxl_core:cxl_query_cmd:539: cxl_mem mem1: Query IOCTL
[ 14.450458] cxl_core:cxl_query_cmd:539: cxl_mem mem1: Query IOCTL
[ 14.452307] cxl_core:cxl_send_cmd:644: cxl_mem mem1: Send IOCTL
[ 14.453686] cxl_core:handle_mailbox_cmd_from_user:602: cxl_pci 0000:0e:00.0: Submitting Get FW Info command for user
[ 14.453686] opcode: 200
[ 14.453686] size: 0
[ 14.460453] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0e:00.0: Sending command: 0x0201
[ 14.462313] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0e:00.0: Doorbell wait took 0ms
[ 14.464166] cxl_pci:__cxl_pci_mbox_send_cmd:310: cxl_pci 0000:0e:00.0: Mailbox background operation (0x0201) started
[ 14.466536] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0e:00.0: Sending command: 0x0200
[ 14.468380] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0e:00.0: Doorbell wait took 0ms
{
"memdev":"mem1",
"pmem_size":"1024.00 MiB (1073.74 MB)",
"serial":"0",
"host":"0000:0e:00.0",
"firmware":{
"num_slots":2,
"active_slot":1,
"online_activate_capable":true,
"slot_1_version":"BWFW VERSION 0",
"fw_update_in_progress":true,
"remaining_size":"48.83 MiB (51.20 MB)"
}
}
cxl memdev: cmd_update_fw: firmware update started on 1 mem device
[ 17.484680] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0e:00.0: Sending command: 0x0005
[ 17.486993] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0e:00.0: Doorbell wait took 0ms
[ 17.489510] cxl_pci:cxl_pci_mbox_send:451: cxl_pci 0000:0e:00.0: Mailbox background operation (0x0201) aborted
[ 17.492476] cxl_pci:__cxl_pci_mbox_send_cmd:262: cxl_pci 0000:0e:00.0: Sending command: 0x4400
[ 17.494937] cxl_pci:cxl_pci_mbox_wait_for_doorbell:74: cxl_pci 0000:0e:00.0: Doorbell wait took 0ms
[ 17.497598] cxl_pci:__cxl_pci_mbox_send_cmd:306: cxl_pci 0000:0e:00.0: Sanitization operation started
[ 25.682631] cxl_pci:cxl_mbox_sanitize_work:164: cxl_pci 0000:0e:00.0: Sanitization operation ended

Thanks!

Davidlohr Bueso (3):
cxl/pci: lockless background synchronous polling
cxl/mbox: support aborting the current background operation
cxl/pci: rename cxl_mbox_background_complete()

drivers/cxl/core/mbox.c | 1 +
drivers/cxl/cxlmem.h | 14 +++
drivers/cxl/pci.c | 189 ++++++++++++++++++++++++++++------------
include/cxl/mailbox.h | 2 +
4 files changed, 152 insertions(+), 54 deletions(-)

--
2.46.1