Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

From: Stefan Agner
Date: Wed Dec 05 2018 - 17:59:56 EST


On 05.12.2018 19:41, Nick Desaulniers wrote:
> On Wed, Dec 5, 2018 at 10:40 AM Ard Biesheuvel
> <ard.biesheuvel@xxxxxxxxxx> wrote:
>>
>> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <natechancellor@xxxxxxxxx> wrote:
>> >
>> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
>> > > (+ Arnd)
>> > >
>> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <natechancellor@xxxxxxxxx> wrote:
>> > > >
>> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
>> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <natechancellor@xxxxxxxxx> wrote:
>> > > > > >
>> > > > > > This flag is not supported by lld:
>> > > > > >
>> > > > > > ld.lld: error: unknown argument: --pic-veneer
>> > > > > >
>> > > > > > Signed-off-by: Nathan Chancellor <natechancellor@xxxxxxxxx>
>> > > > >
>> > > > > Hi Nate,
>> > > > >
>> > > > > Does this mean ld.lld is guaranteed to produce position independent
>> > > > > veneers if you build kernels that are bigger than the typical range of
>> > > > > a relative branch?
>> > > > >
>> > > >
>> > > > Hi Ard,
>> > > >
>> > > > Honestly, I'm not quite sure. I saw your commit that introduced this
>> > > > flag and I wasn't quite sure what to make of it for lld. What
>> > > > configuration would I use to verify and what would I check for?
>> > > >
>> > >
>> > > Try building allyesconfig, and check the resulting binary for veneers
>> > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
>> > > them). These veneers should not take the [virtual] address of the
>> > > branch target directly, but take a PC relative offset (as in the
>> > > example in the commit log of that patch you are referring to)
>> > >
>> >
>> > Alright, compiling with allyesconfig is a little rough at the moment
>> > (bug reports I will file in due time) but I was able to do it. Here's
>> > the disassembly specifically for the functions you had in your commit,
>> > my assembly knowledge is pretty much non-existent unfortunately so I am
>> > not sure what to make of it (it doesn't look like there is a virtual
>> > address for pc in that mix?). I am happy to provide any more information
>> > that is needed.
>> >
>> > c03030cc <__enable_mmu>:
>> > c03030cc: e3c00002 bic r0, r0, #2
>> > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
>> > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
>> > c03030d8: e3a05051 mov r5, #81 ; 0x51
>> > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
>> > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
>> > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
>> > c03030e8: e320f000 nop {0}
>> > c03030ec: e320f000 nop {0}
>> > c03030f0: e320f000 nop {0}
>> > c03030f4: e320f000 nop {0}
>> > c03030f8: e320f000 nop {0}
>> > c03030fc: e320f000 nop {0}
>> >
>> > c0300000 <__turn_mmu_on>:
>> > c0300000: e1a00000 nop ; (mov r0, r0)
>> > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
>> > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
>> > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
>> > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
>> > c0300014: e1a03003 mov r3, r3
>> > c0300018: e1a0300d mov r3, sp
>> > c030001c: e1a0f003 mov pc, r3
>> >
>>
>> Thanks Nate.
>>
>> So these functions no longer appear to reside far away from each
>> other, so there no veneer has been emitted.
>>
>> What we're looking for are veneers, i.e., snippets inserted by the
>> linker that serve as a trampoline so a branch target that is far away
>> can be reached.
>>
>> If no symbols exist with 'veneer' in their name *, it might make sense
>> to rebuild the kernel as Thumb2, which has a branching range of only 8
>> MB (as opposed to 16 MB for ARM mode)
>
> Heh, Arnd and I were just talking about this yesterday. Is there a
> config that sets Thumb2 mode for the kernel?
>

Yes there is CONFIG_THUMB2_KERNEL, and it works also with LLVM/Clang.

However, it sometimes leads to surprising issues, like I just
encountered a few days ago:

https://lore.kernel.org/linux-pci/20181126161645.8177-1-stefan@xxxxxxxx/

--
Stefan

>>
>> * I have no idea whether lld names its veneers like this, or even at all