[PATCH 1/2] mm/page_counter: avoid integer overflow in effective_protection()

[Ridong Chen]

From: Ridong Chen <chenridong@xxxxxxxxxx>

effective_protection() distributes a parent's protection among its
children with two proportional calculations:

	protected * parent_effective / siblings_protected

and, for recursive protection:

	(parent_effective - siblings_protected) * (usage - protected)
		/ (parent_usage - siblings_protected)

All operands are page counts in unsigned long, and the multiplication is
done at unsigned long width before the division. On systems with >= 16TB
RAM the product can exceed 2^64 and wrap, yielding a bogus effective
protection value and silently breaking memory.min/low enforcement. This
was reported by the sashiko review tool [1].

Use mul_u64_u64_div_u64() for both expressions, which performs the
multiply in a 128-bit intermediate before dividing. The divisors are
non-zero on the paths that reach them: siblings_protected >
parent_effective in the first case and parent_usage > siblings_protected
in the second.

[1] https://sashiko.dev/#/patchset/20260826133054.88529-1-ridong.chen@xxxxxxxxx?part=1

Fixes: bc50bcc6e00b ("mm: memcontrol: clean up and document effective low/min calculations")
Fixes: 8a931f801340 ("mm: memcontrol: recursive memory.low protection")
Cc: stable@xxxxxxxxxxxxxxx
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ridong Chen <chenridong@xxxxxxxxxx>
---
 mm/page_counter.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/mm/page_counter.c b/mm/page_counter.c
index 661e0f2a5127..311153b0e002 100644
--- a/mm/page_counter.c
+++ b/mm/page_counter.c
@@ -8,6 +8,7 @@
 #include <linux/page_counter.h>
 #include <linux/atomic.h>
 #include <linux/kernel.h>
+#include <linux/math64.h>
 #include <linux/string.h>
 #include <linux/sched.h>
 #include <linux/bug.h>
@@ -356,7 +357,8 @@ static unsigned long effective_protection(unsigned long usage,
 	 * otherwise get a smaller chunk than what they claimed.
 	 */
 	if (siblings_protected > parent_effective)
-		return protected * parent_effective / siblings_protected;
+		return mul_u64_u64_div_u64(protected, parent_effective,
+					   siblings_protected);
 
 	/*
 	 * Ok, utilized protection of all children is within what the
@@ -399,9 +401,9 @@ static unsigned long effective_protection(unsigned long usage,
 	    usage > protected) {
 		unsigned long unclaimed;
 
-		unclaimed = parent_effective - siblings_protected;
-		unclaimed *= usage - protected;
-		unclaimed /= parent_usage - siblings_protected;
+		unclaimed = mul_u64_u64_div_u64(parent_effective - siblings_protected,
+						usage - protected,
+						parent_usage - siblings_protected);
 
 		ep += unclaimed;
 	}
-- 
2.34.1