[PATCH v1 0/1] limit the i40e msix vectors based on housekeeping CPUs

[Nitesh Narayan Lal]

Issue
=====
With the current implementation at the time of i40e_init_msix(), i40e
creates vectors only based on the number of online CPUs. This would
be problematic for RT setup that includes a large number of isolated
but very few housekeeping CPUs. This is because in those setups
an attempt to move all IRQs from isolated to housekeeping CPUs may
easily fail due to per CPU vector limit.

Setup For The Issue
===================
I have triggered this issue on a setup that had a total of 72
cores among which 68 were isolated and only 4 were left for
housekeeping tasks. I was using tuned's realtime-virtual-host profile
to configure the system. However, Tuned reported the error message
'Failed to set SMP affinity of IRQ xxx to '00000040,00000010,00000005':
[Errno 28] No space left on the device' for several IRQs in tuned.log.
Note: There were other IRQs as well pinned to the housekeeping CPUs that
      were generated by other drivers.

Fix
===
- In this proposed fix I have replaced num_online_cpus in i40e_init_msix()
  with the number of housekeeping CPUs.
- The reason why I chose to include both HK_FLAG_DOMAIN & HK_FLAG_WQ is
  because we would also need IRQ isolation with something like systemd's
  CPU affinity.


Testing
=======
To test this change I had added a tracepoint in i40e_init_msix() to
find the number of CPUs derived for vector creation with and without
tuned's realtime-virtual-host profile. As per expectation with the profile
applied I was only getting the number of housekeeping CPUs and all
available CPUs without it.


Nitesh Narayan Lal (1):
  i40e: limit the msix vectors based on housekeeping CPUs

 drivers/net/ethernet/intel/i40e/i40e_main.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

--