32-bit mmap in a 64-bit process

[LuboÅ DoleÅel]

Hello,

I'm looking for a way of mmaping memory inside a 64-bit process that 
would fit in a 32-bit address.

The reasons for this are a bit unusual. The idea is to run both 64-bit 
and 32-bit macOS applications in 64-bit Linux host processes (Darling), 
because it makes several things easier.

I already have it working, but I'm still not very happy about mmap(). 
After the process is switched into 32-bit mode (by switching segment 
selectors), it makes direct 32-bit system calls via int $0x80.

* Unfortunately, calling SYS_mmap2 via $0x80 doesn't ensure the address 
fits in first 4GB of virtual memory. It just causes the upper 32-bits of 
the address to be cut off, so the return value becomes invalid.

* MAP_32BIT is cr**. The man page acknowledges it's actually MAP_31BIT 
(the allocation is placed within the lowest 2GB) and it gets worse after 
you read the kernel source code, where you learn it's kind of MAP_30BIT, 
because all mappings are made in the 1GB-2GB range.

* Neither of ADDR_LIMIT_32BIT and ADDR_LIMIT_3GB personalities have any 
effect whatsoever. Grepping the kernel source code shows these 
personalities only have effect on Alpha and maybe ARM. Sadly, there is 
no mention of this in the documentation. I would really love to see 
either of them on x86, but I saw a patch implementing them rejected on 
this mailing list.

* I have seen advice to provide a hint address to mmap() that fits 
within <4GB, but I don't see how this could work. The hint is only taken 
into account if there is no mapping at that address yet, otherwise it 
just picks up a random 64-bit address.

Do you have any advice how to proceed?

Thanks!

Lubos Dolezel

Attachment: smime.p7s
Description: Elektronicky podpis S/MIME