[PATCH 0/2] lib/math: rational: fix semiconvergent comparison overflow
From: 961222243021 SHYAM J
Date: Sun Oct 11 2026 - 04:26:58 EST
This series fixes an unsigned-long overflow in the semiconvergent
comparison in rational_best_approximation() and adds five KUnit boundary
cases.
For input 1/ULONG_MAX with maximum numerator 1 and maximum denominator
ULONG_MAX - 1, the current helper returns 0/1 instead of the closer
1/(ULONG_MAX - 1). Doubling the final term t wraps. When t exceeds
ULONG_MAX / 2, its mathematical double exceeds any representable a, so
select the semiconvergent before evaluating the multiplication.
Validation:
- x86-64 KUnit kernel in QEMU: 13/13 cases pass with the fix; applying
only the new tests to the original helper gives 10 pass and 3 fail.
- 32-bit standalone build of the actual helper, executed with qemu-i386:
all 13 parameters pass with the fix; three fail without it.
- Independent exact-fraction oracle: 20 full-width boundary cases and
10,000 deterministic small-bound cases pass.
- Standalone GCC -Wall -Wextra -Werror and UBSan reproducer pass.
- checkpatch --no-signoff: zero errors and zero warnings for both patches.
No affected in-tree driver or physical hardware failure has been
established. This is a helper correctness fix for large valid bounds.
The 32-bit check is a standalone helper test, not a booted KUnit kernel.
UML execution was blocked by this environment's ptrace restriction;
the kernel regression tests were run using x86-64 QEMU instead.
The series was developed with AI assistance, disclosed with Assisted-by:
LLM in both commits. It is based on the mainline commit recorded below.
Shyam J (2):
lib/math: rational: avoid overflow when comparing semiconvergents
lib/math: rational: test large semiconvergent final terms
lib/math/rational.c | 4 +++-
lib/math/tests/rational_kunit.c | 11 +++++++++++
2 files changed, 14 insertions(+), 1 deletion(-)
base-commit: a5ebb76233b79db01e82e061173cb73c6d2b5c6b
--
2.51.1