Re: failed to allocate device WQ

[Leon Romanovsky]

On Fri, Dec 20, 2024 at 05:10:32PM +0000, Holger Kiehl wrote:
> Hello,
> 
> since upgrading from kernel 6.10 to 6.11 (also 6.12) one Infiniband
> card sometimes hits this error:
> 
>    kernel: workqueue: Failed to create a rescuer kthread for wq "ipoib_wq": -EINTR
>    kernel: ib0: failed to allocate device WQ
>    kernel: mlx5_1: failed to initialize device: ib0 port 1 (ret = -12)
>    kernel: mlx5_1: couldn't register ipoib port 1; error -12
> 
> The system has two cards:
> 
>    41:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]
>    c4:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]
> 
> If that happens one cannot use that card for TCP/IP communication. It does
> not always happen, but when it does it always happens with the second
> card mlx5_1. Never with mlx5_0. This happens on four different systems.
> 
> Any idea what I can do to stop this from happening?

It is not related to the FW but to how your system loads kernel modules.

This merged PR in rdma-core probably fixes it.
* Ensure RDMA service loads modules in initrd - https://github.com/linux-rdma/rdma-core/pull/1481

Thanks

> 
> Regards,
> Holger
> 
> PS: Firmware for both cards is 20.41.1000