Re: [PATCH] selftests/mm: fix mlock2 errno handling and false PASS on ENOSYS
From: Gregory Price
Date: Wed Sep 23 2026 - 17:27:38 EST
On Wed, Sep 23, 2026 at 06:26:48PM +0900, Park Tae-sun wrote:
> While inspecting selftests/mm syscall wrappers, I noticed that mlock2_()
> in mlock2.h handles the syscall return value differently from other
> wrappers:
>
> int ret = syscall(__NR_mlock2, start, len, flags);
> if (ret) {
> errno = ret;
> return -1;
> }
>
> Commit 1ddae9d67ee1 ("selftests/mm/mlock: print error on failure")
> introduced this intending to make mlock2_() behave like libc by setting
> errno and returning -1. However, glibc syscall(2) already returns -1 on
> failure and sets positive errno. Assigning "errno = ret;" overwrites
> errno with -1.
>
> To verify this, mlock2 was disabled in the kernel (via sys_ni_syscall) to
> return -ENOSYS. Testing revealed two interrelated defects:
>
> 1. In the unmodified test, mlock2_() clobbered errno to -1. The check
> "if (ret && errno == ENOSYS)" in main() was bypassed, resulting in an
> immediate crash in the first test:
>
> ~ # ./mlock2-tests
> TAP version 13
> 1..15
> Bail out! mlock2(0): Unknown error -1
> # Planned tests != run tests (15 != 0)
> # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:0 error:0
> (exit code: 1 - FAIL)
>
> 2. After restoring mlock2_() to directly return syscall(), errno correctly
> retained ENOSYS (38), entering the ENOSYS check in main(). However, it
> then called ksft_finished():
>
> ~ # ./mlock2-tests
> TAP version 13
> # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:0 error:0
> ~ # echo $?
> 0
>
> Because ksft_set_plan() had not been called yet (ksft_plan == 0) and
> zero tests ran (ksft_pass == 0), ksft_finished() evaluated 0 == 0 as
> success and exited with KSFT_PASS (code 0) without any TAP skip header.
>
> Fix both issues by:
> 1. Returning the syscall() result directly in mlock2_() so that errno is
> preserved.
> 2. Calling ksft_exit_skip() on ENOSYS so unsupported kernels report a TAP
> skip ("1..0 # SKIP ...") and exit with KSFT_SKIP (code 4).
>
> Verification on the mlock2-disabled kernel:
>
> ~ # ./mlock2-tests
> TAP version 13
> 1..0 # SKIP mlock2() syscall is not supported
> ~ # echo $?
> 4
>
> Re-enabling mlock2 in the kernel confirmed all 15 tests pass cleanly:
>
> ~ # ./mlock2-tests
> TAP version 13
> 1..15
> ok 1 test_mlock_lock: Locked
> ...
> ok 15 test_mlockall_future_droppable: droppable memory not locked
> # Totals: pass:15 fail:0 xfail:0 xpass:0 skip:0 error:0
> ~ # echo $?
> 0
>
> Fixes: 1ddae9d67ee1 ("selftests/mm/mlock: print error on failure")
> Fixes: 65c89684896d ("selftests/mm: mlock2-tests: conform test to TAP format output")
> Signed-off-by: Park Tae-sun <ts930@xxxxxxxxx>
Reviewed-by: Gregory Price <gourry@xxxxxxxxxx>