Re: [PATCH v15 8/8] docs: add io_queue flag to isolcpus

[Aaron Tomlin]

On Wed, Jul 15, 2026 at 12:25:35PM +0200, Sebastian Andrzej Siewior wrote:
> On 2026-05-21 19:29:56 [-0400], Aaron Tomlin wrote:
> > From: Daniel Wagner <wagi@xxxxxxxxxx>
> > 
> > The io_queue flag informs multiqueue device drivers where to place
> > hardware queues. Document this new flag in the isolcpus
> > command-line argument description.
> > 
> > Signed-off-by: Daniel Wagner <wagi@xxxxxxxxxx>
> > Reviewed-by: Hannes Reinecke <hare@xxxxxxx>
> > [atomlin:
> >     - Refined io_queue kernel parameter documentation
> >     - Removed an inaccurate claim in the documentation stating
> >       that io_queue takes precedence over managed_irq]
> > Signed-off-by: Aaron Tomlin <atomlin@xxxxxxxxxxx>
> 
> From high level, this looks like managed_irq with the exception that you
> can limit the number of assigned interrupts while managed_irq is best
> effort. Is this true or do I oversimplify something?

Hi Sebastian,

Thank you for reviewing the patch set and for your feedback.

Yes. Though the underlying mechanisms differ in a fundamental way between
the two flags:

isolcpus=managed_irq (post-allocation, best-effort):

    managed_irq operates downstream at the IRQ affinity level (i.e.,
    kernel/irq/manage.c and kernel/irq/cpuhotplug.c). It attempts to steer
    the effective affinity of already-allocated IRQ vectors away from
    isolated CPUs. However, when a multiqueue storage or networking driver
    allocates one hardware queue per CPU, the affinity mask for an isolated
    CPU's queue contains only that specific CPU. Because the intersection
    between the queue's mask and the housekeeping CPUs is empty,
    managed_irq falls back to assigning the interrupt to the isolated CPU.
    It is therefore inherently "best effort" and cannot guarantee isolation
    for 1-queue-per-CPU devices.

isolcpus=io_queue (pre-allocation, strict constraint):

    io_queue operates upstream during device probe, MSI-X vector
    calculation, and block-multiqueue topology setup
    (irq_calc_affinity_vectors(), irq_create_affinity_masks(), and blk-mq).
    Rather than attempting to steer vectors after allocation, it caps the
    total number of hardware queues and MSI-X vectors to match the weight
    of the online housekeeping CPUs. Isolated CPUs are strictly excluded
    during initial vector mask generation (group_mask_cpus_evenly()). I/O
    requests submitted from isolated CPUs are routed through shared
    submission queues mapped onto housekeeping cores, ensuring completion
    interrupts land exclusively on housekeeping CPUs.

In summary, while managed_irq influences where already-created interrupts
land, io_queue restricts how many queues are created and how the hardware
topology is constructed.

I hope this helps.

> If so, can this be integrated?

I do not want managed_irq unconditionally enforced vector capping. Instead
of creating a standalone HK_TYPE_IO_QUEUE housekeeping flag, the strict
vector capping and mask restriction could be integrated under an opt-in
flag?

How about an opt-in flag:

  - isolcpus=managed_irq
        Keeps best-effort post-allocation IRQ steering without capping
        hardware queues

  - isolcpus=managed_irq_strict (or managed_irq:strict)
        Enables strict pre-allocation queue capping (i.e.,
        irq_calc_affinity_vectors() and blk_mq queue limits)

If the preference is to avoid adding a new top-level isolcpus= flag, I am
more than happy to refactor io_queue into a strict sub-parameter under
managed_irq (e.g., isolcpus=managed_irq:strict). This probably makes more
sense.

> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 4d0f545fb3ec..fb828bb60b9e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -2815,6 +2814,31 @@ Kernel parameters
> >  			  housekeeping CPUs has no influence on those
> >  			  queues.
> >  
> > +			io_queue
> > +			  Applicable to managed IRQs only. Restrict
> > +			  multiqueue hardware queue allocation to online
> > +			  housekeeping CPUs. This guarantees that all
> > +			  managed hardware completion interrupts are routed
> > +			  exclusively to housekeeping cores, shielding
> > +			  isolated CPUs from I/O interruptions even if they
> > +			  initiated the request.
> > +
> > +			  Note: Using io_queue restricts the number of
> > +			  allocated hardware queues to match the number of
> > +			  housekeeping CPUs. This prevents MSI-X vector
> > +			  exhaustion and forces isolated CPUs to share
> > +			  submission queues.
> > +
> > +			  Note: Offlining housekeeping CPUs which serve
> > +			  isolated CPUs will fail. The isolated CPUs must
> > +			  be offlined before offlining the housekeeping
> > +			  CPUs.
> > +
> > +			  Note: When I/O is submitted by an application on
> > +			  an isolated CPU, the hardware completion
> > +			  interrupt is handled entirely by a housekeeping
> > +			  CPU.
> 
> This is named io_queue but it does affect I/O in general unless the
> driver uses managed_irqs. This may or may not be clear to the user.
> So the io_queue might not be the perfect name given that affects alles
> devices in the system. We do have just I/O devices using it right now.
> 
> I think this still deserves an explanation in
> Documentation/core-api/irq/managed_irq.rst like how to use it, what is
> the expected outcome and how does it interact with the managed_irq
> argument. Does it make sense to use both, is it exclusive or, what would
> be the use case for both arguments.

Regarding the name io_queue, you are correct that it specifically affects
devices employing managed IRQs (i.e., NVMe, SCSI, and blk-mq devices).
The name was chosen because its primary operational objective is to shield
real-time and latency-sensitive isolated cores from multiqueue storage and
block I/O completion interrupts. Non-managed IRQ devices continue to be
governed by standard irqaffinity settings and irqbalance.

I would be happy to provide clear usage guidelines based on the above, if
we can agree on the new proposal, irq_managed:strict?


Kind regards,
-- 
Aaron Tomlin