[PATCH v2 0/3] tools/nolibc: fix readdir_r() and the FD_* macros on 64-bit

[Danish Khateeb]

Two bugs that only show up on 64-bit architectures:

- readdir_r() truncates the directory offset to an int. On ext4 it
  fails within the first few entries of almost every directory (since
  v6.19), or returns the next-to-last entry twice and never the last
  one (v6.15 to v6.18). Patch 1.

- FD_SET(), FD_CLR() and FD_ISSET() build their masks from an int, so
  they use the wrong bits for fds 31-63 of each 64-bit word, and
  select() on fd 40 fails with EBADF. Patch 2, with a test in patch 3.

There is no selftest for patch 1: it needs a directory with large
offsets, and nolibc-test only lists /proc/self, whose offsets are
small. I tested it with a small program that lists a directory with
readdir_r() instead.

Changes in v2:
- Dropped Cc: stable from patches 1 and 2, which are otherwise
  unchanged.
- Patch 3: dropped the select() test, and moved the FD_* test to the
  stdlib tests, with its helper in the same order as the test cases
  (Thomas). It is also shorter: FD_CLR() is now checked on a set
  filled with memset().
- Rebased on nolibc/for-next 6f114ae7b600.

v1: https://lore.kernel.org/all/20260924015222.31693-1-danishkhateeb03@xxxxxxxxx/

Testing, with GCC 16.2, on x86_64 and i386 natively and on arm, arm64
and sparc64 under qemu-user:

- nolibc-test, all tests: no failures before or after the series. The
  only difference is the new fd_set test, which passes, also with clang
  on x86_64 and against glibc (make libc-test).

- Without patch 2, fd_set fails on the 64-bit architectures: with the
  default UBSan flags the run stops at fd_set on the sanitizer trap,
  and without them the test reports [FAIL] (checked on x86_64 and
  arm64). It passes on i386 and arm, which are not affected.

- The readdir_r() program, on ext4, on x86_64, arm64 and sparc64:
  before, it fails on /etc after 4 of 215 entries, /usr/include after
  1 of 1467 and /usr/lib after 2 of 6229. After, it lists as many
  entries as glibc. A native i386 build was already correct.

- Built for x86_64 against nolibc from before commit 4ada5679f18d, the
  same program lists a directory holding "..", "code" and "." as "..",
  "code", "code".

Danish Khateeb (3):
  tools/nolibc: fix readdir_r() with 64-bit directory offsets
  tools/nolibc: fix FD_SET(), FD_CLR() and FD_ISSET() on 64-bit
  selftests/nolibc: test the FD_* macros

 tools/include/nolibc/dirent.h                |  7 ++++---
 tools/include/nolibc/sys/select.h            |  6 +++---
 tools/testing/selftests/nolibc/nolibc-test.c | 21 ++++++++++++++++++++
 3 files changed, 28 insertions(+), 6 deletions(-)


base-commit: 6f114ae7b6006ca7cf72e3e3348d263bbfec4525
-- 
2.55.0