Re: [RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Costa Shulyupin]

Hello Michal.

Isolation of CPUs from blk_mq_hw_ctx during boot is already handled on
call hierarchy:
...
        nvme_probe()
                nvme_alloc_admin_tag_set()
                        blk_mq_alloc_queue()
                                blk_mq_init_allocated_queue()
                                        blk_mq_map_swqueue()

blk_mq_map_swqueue() performs:
for_each_cpu(cpu, hctx->cpumask) {
        if (cpu_is_isolated(cpu))
                cpumask_clear_cpu(cpu, hctx->cpumask);
}

static inline bool cpu_is_isolated(int cpu)
{
        return !housekeeping_test_cpu(cpu, HK_TYPE_DOMAIN) ||
                !housekeeping_test_cpu(cpu, HK_TYPE_TICK) ||
               cpuset_cpu_is_isolated(cpu);
}

cpu_is_isolated() is introduced by  3232e7aad11e5.

Thanks,
Costa


On Fri, 15 Nov 2024 at 17:45, Michal Koutný <mkoutny@xxxxxxxx> wrote:
>
> Hello.
>
> On Fri, Nov 08, 2024 at 07:48:30AM GMT, Costa Shulyupin <costa.shul@xxxxxxxxxx> wrote:
> > Cgroups allow configuring isolated_cpus at runtime.
> > However, blk-mq may still use managed interrupts on the
> > newly isolated CPUs.
> >
> > Rebuild hctx->cpumask considering isolated CPUs to avoid
> > managed interrupts on those CPUs and reclaim non-isolated ones.
> >
> > The patch is based on
> > isolation: Exclude dynamically isolated CPUs from housekeeping masks:
> > https://lore.kernel.org/lkml/20240821142312.236970-1-longman@xxxxxxxxxx/
>
> Even based on that this seems incomplete to me the CPUs that are part of
> isolcpus mask on boot time won't be excluded from this?
> IOW, isolating CPUs from blk_mq_hw_ctx would only be possible via cpuset
> but not "statically" throught the cmdline option, or would it?
>
> Thanks,
> Michal
>
> (-Cc: lizefan.x@xxxxxxxxxxxxx)