Re: ppc elf_map breakage with MAP_FIXED_NOREPLACE
From: Anshuman Khandual
Date: Tue Jan 23 2018 - 10:58:55 EST
On 01/23/2018 06:15 PM, Michal Hocko wrote:
> On Tue 23-01-18 16:55:18, Anshuman Khandual wrote:
>> On 01/17/2018 01:37 PM, Michal Hocko wrote:
>>> On Thu 11-01-18 15:38:37, Anshuman Khandual wrote:
>>>> On 01/09/2018 09:43 PM, Michal Hocko wrote:
>>> [...]
>>>>> Did you manage to catch _who_ is requesting that anonymous mapping? Do
>>>>> you need a help with the debugging patch?
>>>>
>>>> Not yet, will get back on this.
>>>
>>> ping?
>>
>> Hey Michal,
>>
>> Missed this thread, my apologies. This problem is happening only with
>> certain binaries like 'sed', 'tmux', 'hostname', 'pkg-config' etc. As
>> you had mentioned before the map request collision is happening on
>> [10030000, 10040000] and [10030000, 10040000] ranges only which is
>> just a single PAGE_SIZE. You asked previously that who might have
>> requested the anon mapping which is already present in there ? Would
>> not that be the same process itself ? I am bit confused.
>
> We are early in the ELF loading. If we are mapping over an existing
> mapping then we are effectivelly corrupting it. In other words exactly
> what this patch tries to prevent. I fail to see what would be a relevant
> anon mapping this early and why it would be colliding with elf
> segements.
>
>> Would it be
>> helpful to trap all the mmap() requests from any of the binaries
>> and see where we might have created that anon mapping ?
>
> Yeah, that is exactly what I was suggesting. Sorry for not being clear
> about that.
>
Tried to instrument just for the 'sed' binary and dont see any where
it actually requests the anon VMA which got hit when loading the ELF
section which is strange. All these requested flags here already has
MAP_FIXED_NOREPLACE (0x100000). Wondering from where the anon VMA
actually came from.
cat /sys/kernel/debug/tracing/trace | grep '10030000 10040000'
sed-7579 [008] .... 10.358521: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-7583 [060] .... 10.358640: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8520 [059] .... 14.955216: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8802 [040] .... 23.063756: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8804 [051] .... 23.064434: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8828 [051] .... 23.334103: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8836 [066] .n.. 23.369308: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8859 [013] .... 23.436563: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8868 [020] .... 23.484949: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8916 [066] .... 23.692761: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8936 [065] .... 23.860220: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8941 [066] .... 23.864280: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8946 [006] .... 23.868742: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8952 [056] .... 23.906065: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8969 [044] .... 24.199167: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8975 [005] .... 24.212314: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8980 [046] .... 24.215694: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-8987 [028] .... 24.223261: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9001 [016] .... 24.247962: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9004 [048] .... 24.250521: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9007 [017] .... 24.253165: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9010 [005] .... 24.255651: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9013 [030] .... 24.258122: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9016 [004] .... 24.260814: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9020 [029] .... 24.266019: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9026 [003] .n.. 24.269327: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9029 [053] .... 24.272223: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9032 [030] .... 24.274914: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9035 [068] .... 24.277492: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9038 [007] .... 24.280134: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9043 [065] .... 24.286741: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9046 [060] .... 24.289189: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9049 [018] .... 24.291915: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9052 [068] .... 24.294782: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9056 [044] .... 24.299412: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9059 [068] .... 24.302153: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9062 [055] .... 24.304719: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9065 [046] .... 24.307286: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9069 [029] .... 24.311957: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9072 [010] .... 24.315013: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9075 [002] .... 24.317803: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9078 [030] .... 24.320709: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9081 [011] .... 24.323806: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9085 [042] .... 24.329019: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9091 [049] .... 24.337911: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9100 [023] .... 24.392637: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9103 [058] .... 24.395753: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9106 [066] .... 24.398322: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9109 [023] .... 24.401226: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9112 [057] .... 24.404415: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9115 [045] .... 24.406961: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9125 [007] .... 24.460685: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9128 [021] .... 24.463883: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
sed-9284 [046] .... 340.883633: do_mmap: comm sed range [10030000 10040000] prot 3 flags 101802
Errors while loading ELF sections.
$dmesg | grep 'sed requested'
[ 10.358608] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 10.358646] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 14.955315] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.063862] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.064445] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.334212] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.369408] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.436664] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.485034] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.692866] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.860316] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.864333] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.868912] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 23.906131] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.199315] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.212413] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.215787] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.223368] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.248050] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.250594] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.253236] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.255709] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.258191] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.260887] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.266149] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.269456] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.272336] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.275035] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.277572] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.280204] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.286818] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.289273] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.291999] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.294861] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.299508] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.302229] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.304801] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.307371] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.312062] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.315121] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.317913] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.320821] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.323901] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.329124] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.338009] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.392780] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.395842] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.398442] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.401325] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.404494] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.407072] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.460794] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 24.463986] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon
[ 340.883717] sed requested [10030000, 10040000] mapped [10030000, 10040000] 100073 anon