Re: [PATCH v17 00/19] PCI/CXL: Add CXL reset support for Type 2 devices

From: Srirangan Madhavan

Date: Fri Nov 20 2026 - 03:16:26 EST


On 11/17/26 9:28 AM, Alison Schofield wrote:
Hi folks,

This series adds CXL Reset for Type 2 devices through the existing PCI
reset_method and reset sysfs attributes. It disables CXL.cache, performs
supported cache writeback, initiates CXL Reset with Memory Clear, and
restores cached HDM decoder and CXL Device DVSEC state before PCI use
resumes. Active HDM ranges are reserved and CPU caches invalidated before
and after reset. The method remains limited to eligible single-function
devices with both CXL.cache and CXL.mem enabled. CXL bus reset restores
the same cached state and flushes the host and device caches around the
reset.

The reset sequence follows the CXL 4.0 specification [4], including
CXL.cache disable, optional cache writeback, CXL Reset Memory Clear,
ResetComplete polling, and ResetError reporting. Sections 9.7.2 and 9.7.3
define the completion timeout encodings and reset access restrictions.

The userspace ABI is the existing PCI reset interface:

/sys/bus/pci/devices/.../reset_method
/sys/bus/pci/devices/.../reset

Userspace selects "cxl_reset" in reset_method and triggers the reset
through reset. Following Dan's v6 feedback, this uses the PCI interface
instead of adding a memdev-specific sysfs ABI.

For sysfs reset, the generic PCI path invokes the driver's reset_prepare()
callback when provided and disables PCI I/O, memory decoding, and bus
mastering. The CXL path reserves affected HDM ranges exclusively and
blocks IOMMU translations where applicable. It does not offline memory
or revoke mappings: userspace is expected to offline CXL.mem users before
reset. The reservation enforces this for claimed ranges by failing the
reset while a conflicting resource is claimed [9].

The global region lock is held from snapshot through restoration. Memory
Clear completion is awaited for the timeout the device advertises, up to
100 seconds, so the lock can be held for that long plus cache flushes.
Hi Srirangan,

Thanks for all the followups on review feedback. I think you answer some
of my question above, 'For sysfs reset...' Could you explain the intended
use cases for this reset method? Like why and when would userspace issue
a CXL reset. In particular, is reset intended to be available before endpoint
and region discovery have completed?


Hi Alison,

I am looking at these types of use cases: a VMM handing a Type 2
accelerator from one guest to another (the vfio-cxl series), where we
need the device's memory cleared and its CXL state reset and restored,
which FLR doesn't give us. Or an admin or a tool resetting a hung or
errored device through reset_method and reset, without taking down the
whole link. In those cases we need the CXL reset. The CXL 4.0 spec says
FLRs do not affect the CXL.cache and CXL.mem protocols, and that errors
in CXL.cache and CXL.mem logic cannot be recovered by an FLR.

My understanding is that it is intended to be available at enumeration,
before endpoint and region discovery. I can fix the gaps in the address
caching mechanism to make it work.

cxl_reset is listed at device add, like the other reset methods. It can
execute once the HDM cache exists, which is right after enumeration.

I see how the series saves state for restoration after reset. I don't see
what ensures discovery has progressed far enough for reset to be safe.


Nothing as of v17. Normalized Addressing is recorded after the HDM cache is
filled, so an early reset can treat a range that isn't a HPA as one.

What I'm thinking of for v18 is a check that doesn't wait for the flag:
also reject an enabled decoder whose range isn't inside a platform CXL
window (the "CXL Window" resources cxl_acpi registers). Until cxl_acpi
has registered them nothing matches, so the reset is refused. The flag
check stays as it is.

Both cxl_reset and cxl_bus reach it through cxl_reset_cache_prepare():

cxl_reset_cache_prepare()
cxl_hdm_quiesce_prepare()
cxl_hdm_ranges_collect() per enabled, non-zero-size decoder:
NORMALIZED flag set -> -EOPNOTSUPP (today)
range not in a CXL window -> -EOPNOTSUPP (new)
cxl_hdm_range_add()
cxl_hdm_ranges_request() reserve the ranges
cxl_hdm_ranges_invalidate() flush host caches

So a reset on a range we can't validate is refused before anything is
reserved or flushed. Does this sounds like a good idea?

I also see DaveJ's cxl_bus deadlock report. My concern is related but
separate. Even without a deadlock, a reset could hold the global region lock
for up to 100 seconds plus cache maintenance and restoration, blocking
discovery and region assembly on unrelated devices. Understanding the intended
usage would help assess whether that disruption is acceptable. Note that my
simple concern is a user issuing those cxl_reset's on repeat and blocking CXL
bringup. Yeah, that's a bad user, but is there any protection against that?

Adding specific questions on patches 11, 13, and 16


I rechecked the code for avoiding the global lock. For v18, I think I can reduce both resets to take the lock briefly twice instead:

prepare: lock -> snapshot, reserve ranges, set reset_pending -> unlock
do reset: no region lock
restore: lock -> re-read the cache, restore, clear reset_pending -> unlock

While reset_pending is set, cxl_decoder_commit() on that device returns
-EBUSY.

This way, the long lock is not held and remaining is scoped to the device being reset: only commits of that device's decoders return -EBUSY during the reset, so a region that includes it can't commit until the reset is done. The two brief windows are still global. I couldn't find a norrower option.

I posted the same proposal with some more details in reply to DaveJ
https://lore.kernel.org/linux-cxl/20261105185917.2667656-1-smadhavan@xxxxxxxxxx/T/#mcea903d781c011fe22abf76ddd16d4f78a3c14f3
Please let me know if this could work.

-- Alison

--
Regards,
Srirangan