Re: [PATCH RFC net-next 2/3] crypto: caam/qi2 - skip isolated CPUs when setting up queue pairs

[Ioana Ciornei]

On Thu, Oct 08, 2026 at 10:49:57AM +0000, Josua Mayer wrote:
> Hi Ioana,
> 
> Thank you for taking a look at my patch!
> 
> On 10/6/26 11:27, Ioana Ciornei wrote:
> > On Wed, Sep 30, 2026 at 06:54:59PM +0200, Josua Mayer wrote:
> >> The driver looks up the affine DPIO and sets up one Rx/Tx queue pair for
> >> each online CPU, up to the number of queues. Response notifications and
> >> NAPI polling are handled on the DPIO object's assigned CPU. All online
> >> may enqueue requests.
> >>
> >> When isolcpus=managed_irq is used to reserve CPUs for latency-sensitive
> >> applications, those CPUs are expected to be free of routine device
> >> interrupts.
> >>
> >> The DPIO driver assigns affine CPUs only from the HK_TYPE_MANAGED_IRQ
> >> housekeeping mask, leaving isolated CPUs without an affine DPIO. This
> >> defers caam qi2 probe indefinitely once DPIO setup reaches an isolated
> >> CPU.
> >>
> >> Restrict queue pairs, notifications and NAPI polling to the online CPUs
> >> in the HK_TYPE_MANAGED_IRQ housekeeping mask, so that isolated CPUs are
> >> skipped and don't defer probe. Isolated CPUs may still enqueue requests,
> >> with responses handled on housekeeping CPUs.
> >
> > Don't you need to also change the for_each_online_cpu() from
> > dpaa2_dpseci_setup() to take into account only the housekeeping
> > managed_irq cores?
> I initially thought no, because of the "Allow all cores to enqueue"
> comment in code.
> >
> >
> > Something like:
> >
> > --- a/drivers/crypto/caam/caamalg_qi2.c
> > +++ b/drivers/crypto/caam/caamalg_qi2.c
> > @@ -5101,7 +5101,7 @@ static int __cold dpaa2_dpseci_setup(struct fsl_mc_device *ls_dev)
> >          }
> >
> >          i = 0;
> > -       for_each_online_cpu(cpu) {
> > +       for_each_cpu_and(cpu, cpu_online_mask, hk_mask) {
> >                  u8 j;
> >
> >                  j = i % priv->num_pairs;
> 
> Correct.
> 
> The loop is doing two things in one: assign tx queue to each cpu,
> and assign rx queue to each cpu.
> 
> Only rx side is an issue for isolcpus. Do you have any idea why each cpu
> got a tx queue assigned, and if this may be changed?
> 
> 
> Here is a copy of the complete loop, with inline comments on the
> control flow:
> 
> > 	i = 0;
> > 	for_each_online_cpu(cpu) {
> > 		u8 j;
> >
> > 		j = i % priv->num_pairs;
> >
> > 		ppriv = per_cpu_ptr(priv->ppriv, cpu);
> > 		ppriv->req_fqid = priv->tx_queue_attr[j].fqid;
> Assigns a tx queue to the cpu, one tx queue may be assigned to
> multiple cores due to modulo above (if i >= num_pairs).
> > 		/*
> > 		 * Allow all cores to enqueue, while only some of them
> > 		 * will take part in dequeuing.
> > 		 */
> > 		if (++i > priv->num_pairs)
> > 			continue;
> After this i <= num_pairs.
> > 		ppriv->rsp_fqid = priv->rx_queue_attr[j].fqid;
> > 		ppriv->prio = j;
> Assigns a rx queue to the cpu.
> This is the case which causes issues with isolcpus, and it should
> be avoided for non-housekeeping cpus.
> > 		dev_dbg(dev, "pair %d: rx queue %d, tx queue %d\n", j,
> > 			priv->rx_queue_attr[j].fqid,
> > 			priv->tx_queue_attr[j].fqid);
> >
> > 		ppriv->net_dev = alloc_netdev_dummy(0);
> > 		if (!ppriv->net_dev) {
> > 			err = -ENOMEM;
> > 			goto err_alloc_netdev;
> > 		}
> > 		cpumask_set_cpu(cpu, priv->clean_mask);
> > 		ppriv->net_dev->dev = *dev;
> >
> > 		netif_napi_add_tx_weight(ppriv->net_dev, &ppriv->napi,
> > 					 dpaa2_dpseci_poll,
> > 					 DPAA2_CAAM_NAPI_WEIGHT);
> > 	}
> dpaa2_caam_enqueue unconditionally accesses ppriv->dpio per-cpu
> and ppriv->req_fqid, not gated by any checks, percpu (any cpu).
> 
> This causes two issues:
> 1. each cpu is still expected to have a dpio object
> 2. each cpu is still expected to have an assigned tx queue
> 
> I don't quite understand how we get into dpaa2_caam_enqueue,
> I suppose it can legitimately happen on any cpu core whatsoever?

I don't have any experience in the crypto area but from the looks of it,
there is nothing limiting on what cpus crypto_aead_encrypt() could run on.

For example, one of crypto_aead_encrypt() callers is the macsec software
implementation. In that case, I would expect that the macsec's
.ndo_start_xmit() callback could be run on a non-housekeeping, which in
turn means that caam_qi2 needs to be prepared to run the enqueue() from
any cpu.

Ioana