[PATCH v1 4/6] mm: make lru_drain selective where it schedules work

[Gilad Ben-Yossef]

lru drain work is being done by scheduling a work queue on each
CPU, whether it has LRU pages to drain or not, thus creating
interference on isolated CPUs.

This patch uses schedule_on_each_cpu_cond() to schedule the work
only on CPUs where it seems that there are LRUs to drain.

Signed-off-by: Gilad Ben-Yossef <gilad@xxxxxxxxxxxxx>
CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
CC: Tejun Heo <tj@xxxxxxxxxx>
CC: John Stultz <johnstul@xxxxxxxxxx>
CC: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
CC: KOSAKI Motohiro <kosaki.motohiro@xxxxxxxxxxxxxx>
CC: Mel Gorman <mel@xxxxxxxxx>
CC: Mike Frysinger <vapier@xxxxxxxxxx>
CC: David Rientjes <rientjes@xxxxxxxxxx>
CC: Hugh Dickins <hughd@xxxxxxxxxx>
CC: Minchan Kim <minchan.kim@xxxxxxxxx>
CC: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxx>
CC: Christoph Lameter <cl@xxxxxxxxx>
CC: Chris Metcalf <cmetcalf@xxxxxxxxxx>
CC: Hakan Akkan <hakanakkan@xxxxxxxxx>
CC: Max Krasnyansky <maxk@xxxxxxxxxxxx>
CC: Frederic Weisbecker <fweisbec@xxxxxxxxx>
CC: linux-kernel@xxxxxxxxxxxxxxx
CC: linux-mm@xxxxxxxxx
---
 mm/swap.c |   25 ++++++++++++++++++++++++-
 1 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/mm/swap.c b/mm/swap.c
index 5c13f13..ab07b62 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -562,12 +562,35 @@ static void lru_add_drain_per_cpu(struct work_struct *dummy)
 	lru_add_drain();
 }
 
+static bool lru_drain_cpu(int cpu)
+{
+	struct pagevec *pvecs = per_cpu(lru_add_pvecs, cpu);
+	struct pagevec *pvec;
+	int lru;
+
+	for_each_lru(lru) {
+		pvec = &pvecs[lru - LRU_BASE];
+		if (pagevec_count(pvec))
+			return true;
+	}
+
+	pvec = &per_cpu(lru_rotate_pvecs, cpu);
+	if (pagevec_count(pvec))
+		return true;
+
+	pvec = &per_cpu(lru_deactivate_pvecs, cpu);
+	if (pagevec_count(pvec))
+		return true;
+
+	return false;
+}
+
 /*
  * Returns 0 for success
  */
 int lru_add_drain_all(void)
 {
-	return schedule_on_each_cpu(lru_add_drain_per_cpu);
+	return schedule_on_each_cpu_cond(lru_add_drain_per_cpu, lru_drain_cpu);
 }
 
 /*
-- 
1.7.0.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/