[PATCH v4 2/2] PCI: Call local_pci_probe() directly in pci_call_probe() if affined to the right node

[Waiman Long]

Currently pci_call_probe() will unconditionally queue a work function
to perform the actual device probing whenever the device is associated
with a valid NUMA node. The use of work function originally comes from
commit 873392ca514f ("PCI: work_on_cpu: use in drivers/pci/pci-driver.c")
to execute the device probing and allocate memory on the right node
where the device bus is attached to.

This is inefficient to schedule another work function in the same or a
neigboring CPU and wait for its completion when the current task is a
work function itself queued by pci_call_probe() or async_schedule_dev()
that has been affined to the right node already. It will also reduce
the amount of parallelism as noted in [1].

Fix that by calling local_pci_probe() directly if the current task has
been affined to the right NUMA node. Now lockdep dynamic key will only
be used if the current task is a wq kworker but is not affined to the
right node.

[1] https://lore.kernel.org/linux-pci/20251227113326.964-1-guojinhui.liam@xxxxxxxxxxxxx/

Signed-off-by: Waiman Long <longman@xxxxxxxxxx>
---
 drivers/pci/pci-driver.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index bc8c0f061072..70e7b944c097 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -375,6 +375,8 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
 {
 	int error, node, cpu;
 	struct drv_dev_and_id ddi = { drv, dev, id };
+	bool node_invalid, affine_to_node;
+	const struct cpumask *node_cpus;
 
 	/*
 	 * Execute driver initialization on node where the device is
@@ -383,14 +385,24 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
 	 */
 	node = dev_to_node(&dev->dev);
 	dev->is_probed = 1;
+	node_invalid = node < 0 || node >= MAX_NUMNODES || !node_online(node);
+
+	if (node_invalid) {
+		node_cpus = cpu_online_mask;
+		affine_to_node = false;
+	} else {
+		node_cpus = cpumask_of_node(node);
+		/* Check if the current task is affined to the right node */
+		affine_to_node = cpumask_subset(current->cpus_ptr, node_cpus);
+	}
 
 	cpu_hotplug_disable();
 	/*
-	 * Prevent nesting work_on_cpu() for the case where a Virtual Function
-	 * device is probed from work_on_cpu() of the Physical device.
+	 * Prevent nesting queue_work_on() for the case where a Virtual Function
+	 * device is probed from queue_work_on() of the Physical function or
+	 * when the current task is affined to the right node.
 	 */
-	if (node < 0 || node >= MAX_NUMNODES || !node_online(node) ||
-	    pci_physfn_is_probed(dev)) {
+	if (node_invalid || affine_to_node || pci_physfn_is_probed(dev)) {
 		error = local_pci_probe(&ddi);
 	} else {
 		struct pci_probe_arg arg = { .ddi = &ddi };
@@ -417,8 +429,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev,
 		 * targets.
 		 */
 		rcu_read_lock();
-		cpu = cpumask_any_and(cpumask_of_node(node),
-				      housekeeping_cpumask(HK_TYPE_DOMAIN));
+		cpu = cpumask_any_and(node_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN));
 
 		if (cpu < nr_cpu_ids) {
 			struct workqueue_struct *wq = pci_probe_wq;
-- 
2.54.0