Re: [PATCH] mm/mmap: Map MAP_STACK to VM_STACK

From: Waiman Long
Date: Tue Apr 18 2023 - 21:46:32 EST



On 4/18/23 21:36, Hugh Dickins wrote:
On Tue, 18 Apr 2023, Waiman Long wrote:
On 4/18/23 17:18, Andrew Morton wrote:
On Tue, 18 Apr 2023 17:02:30 -0400 Waiman Long <longman@xxxxxxxxxx> wrote:

One of the flags of mmap(2) is MAP_STACK to request a memory segment
suitable for a process or thread stack. The kernel currently ignores
this flags. Glibc uses MAP_STACK when mmapping a thread stack. However,
selinux has an execstack check in selinux_file_mprotect() which disallows
a stack VMA to be made executable.

Since MAP_STACK is a noop, it is possible for a stack VMA to be merged
with an adjacent anonymous VMA. With that merging, using mprotect(2)
to change a part of the merged anonymous VMA to make it executable may
fail. This can lead to sporadic failure of applications that need to
make those changes.
"Sporadic failure of applications" sounds quite serious. Can you
provide more details?
The problem boils down to the fact that it is possible for user code to mmap a
region of memory and then for the kernel to merge the VMA for that memory with
the VMA for one of the application's thread stacks. This is causing random
SEGVs with one of our large customer application.

At a high level, this is what's happening:

 1) App runs creating lots of threads.
 2) It mmap's 256K pages of anonymous memory.
 3) It writes executable code to that memory.
 4) It calls mprotect() with PROT_EXEC on that memory so
    it can subsequently execute the code.

The above mprotect() will fail if the mmap'd region's VMA gets merged with the
VMA for one of the thread stacks.  That's because the default RHEL SELinux
policy is to not allow executable stacks.
Then wouldn't the bug be at the SELinux end? VMAs may have been merged
already, but the mprotect() with PROT_EXEC of the good non-stack range
will then split that area off from the stack again - maybe the SELinux
check does not understand that must happen?

The SELinux check is done per VMA, not a region within a VMA. After VMA merging, SELinux is probably not able to determine which part of a VMA is a stack unless we keep that information somewhere and provide an API for SELinux to query. That can be quite a lot of work. So the easiest way to prevent this problem is to avoid merging a stack VMA with a regular anonymous VMA.

Cheers,
Longman