[RFC PATCH v2 0/4] resctrl: Assign registered devices to resource groups

From: Zhanpeng Zhang

Date: Mon Nov 16 2026 - 02:14:00 EST


This RFC lets resctrl groups include devices as well as CPU tasks. It
uses the RISC-V IOMMU QoS ID extension as the first provider and builds
on Drew Fustini's CBQRI resctrl v12 series [1] on Linux v7.4-rc1.

RISC-V QoS tags requests with a resource control ID (RCID) and a
monitoring counter ID (MCID). On successful translations the IOMMU
supplies the device context's IDs to the I/O bridge for device requests.
Those IDs also tag IOMMU accesses to context-specific metadata, such as
page tables. IOMMU accesses to the device directory, queues and
IOMMU-generated MSIs use the per-IOMMU iommu_qosid register. When the
IOMMU is in DDTP Bare mode, device requests use that register too.
The IOMMU supplies the tags; downstream controllers apply the policy.

In v1, the unit of assignment was an IOMMU group. In v2, the driver
responsible for tagging explicitly registers supported devices with
resctrl. Here that is the RISC-V IOMMU driver, so endpoint drivers such
as NVMe do not need to change. Common resctrl code handles token
lookup and group membership. When a device is assigned, resctrl calls
the driver's set_ids() callback to program its device context. The
devices file remains separate from tasks.

resctrl control group
schemata ---------> CBQRI controller: resource policy
tasks ---------> CPU: task IDs
devices ---------> IOMMU: device context IDs

For example, for an independently tagged PCI device:

mkdir /sys/fs/resctrl/p1
echo pci/0000:00:03.0 > /sys/fs/resctrl/p1/devices
echo pci/0000:00:03.0 > /sys/fs/resctrl/devices
rmdir /sys/fs/resctrl/p1

The devices file appears only if at least one device is registered at
mount time. A device being present in sysfs does not make it assignable.
The IOMMU driver excludes shared hardware contexts and requires known QoS
ID limits before registering a device. It probes those limits only with
DDTP set to Off and all queues disabled and idle, and rejects out-of-range
IDs before changing a device context.

The separate qosid/{rcid,mcid} attributes under the IOMMU class device
configure the global register independently of resctrl.

The v12 dependency provides cache capacity allocation, not monitoring.
Nonzero device MCIDs require a monitoring-capable dependency.

Changes since RFC v1:

* Assign devices rather than IOMMU groups, as Jason and Qinxin
suggested [2]. Drop the IOMMU core group lookup and walk helpers.
* Build on James's driver callback suggestion [2] with explicit
device registration. Handle token lookup in common resctrl rather
than in each architecture, as Drew suggested [3].
* Keep devices separate from tasks so the existing ABI stays
unchanged [4]. Following Drew's runtime visibility suggestion [5],
check for registered devices at mount time.
* Split the global sysfs setting into separate rcid and mcid files
[6], and introduce qosid_lock in the patch that first uses it [7].
Thanks to Drew for both reviews.
* Rebase on CBQRI v12, check implemented QoS ID widths before device
assignment, and test task and device membership in the same
control group.

Feedback would be particularly helpful on:

* The provider registration and lifetime contract, including how to
represent devices that share a tagging context.
* The bus/name token versus using an existing sysfs identity.
* The scope of protection after a command queue failure. If an update
leaves the hardware state uncertain, the current code prevents ID
reuse by blocking group removal and future mounts, including for
groups unrelated to the failed update. Would reserving only the
affected old and target IDs be sufficient?

PASID assignment and recovery after an uncertain hardware update are
outside this RFC.

[1] https://lore.kernel.org/linux-riscv/20261114-dfustini-atl-sc-cbqri-dt-v12-0-6ef1f627072d@xxxxxxxxxx/
[2] https://lore.kernel.org/all/arKx__kOPe4B5gDW@agluck-desk3/T/
[3] https://lore.kernel.org/all/an4CRRXt44%2BPhHgy@x1/
[4] https://lore.kernel.org/all/7cf591e8-e182-447f-9808-d8e328d28f79@xxxxxxx/#t
[5] https://lore.kernel.org/all/anvYGuNF3MC%2F%2FQ94@x1/
[6] https://lore.kernel.org/all/andiub47zsUsL4qh@x1/
[7] https://lore.kernel.org/all/andiWrKhz5rLPVuq@x1/

Zhanpeng Zhang (4):
resctrl: Add registered device assignment
iommu/riscv: Register independently tagged devices with resctrl
iommu/riscv: Expose global QoS IDs in sysfs
selftests/resctrl: Exercise device assignment and IOMMU global IDs

.../ABI/testing/sysfs-class-iommu-riscv-iommu | 27 ++
.../filesystems/resctrl-devices-prototype.rst | 43 ++
MAINTAINERS | 8 +
drivers/iommu/riscv/iommu-bits.h | 8 +
drivers/iommu/riscv/iommu.c | 398 +++++++++++++++++-
drivers/iommu/riscv/iommu.h | 16 +
fs/resctrl/Makefile | 2 +-
fs/resctrl/devices.c | 233 ++++++++++
fs/resctrl/internal.h | 8 +
fs/resctrl/rdtgroup.c | 28 ++
include/linux/resctrl_device.h | 53 +++
tools/testing/selftests/resctrl/Makefile | 2 +
tools/testing/selftests/resctrl/devices.sh | 205 +++++++++
.../testing/selftests/resctrl/iommu_qosid.sh | 119 ++++++
14 files changed, 1143 insertions(+), 7 deletions(-)
create mode 100644 Documentation/ABI/testing/sysfs-class-iommu-riscv-iommu
create mode 100644 Documentation/filesystems/resctrl-devices-prototype.rst
create mode 100644 fs/resctrl/devices.c
create mode 100644 include/linux/resctrl_device.h
create mode 100755 tools/testing/selftests/resctrl/devices.sh
create mode 100755 tools/testing/selftests/resctrl/iommu_qosid.sh


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
prerequisite-patch-id: 492fe81aa4f3c885ac5bb48252def2fdc5527275
prerequisite-patch-id: b4f0bd4bf660c92dbf4ca0c7d871c1550b66df0d
prerequisite-patch-id: a2e36f0788f8e11c708d162dce653351dd99c4f0
prerequisite-patch-id: d1d359c5cf89b2b5018d64ca8f320892f6dbcca2
prerequisite-patch-id: 5b7fea6f1ad8f2137ffe45681d56b83e17f7cb2a
--
2.50.1 (Apple Git-155)