Re: WARNING: kernel stack frame pointer at ffffffff82e03f40 in swapper:0 has bad value (null)
From: Andy Lutomirski
Date: Tue Dec 13 2016 - 11:56:22 EST
On Tue, Dec 13, 2016 at 6:34 AM, Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:
> On Mon, Dec 12, 2016 at 05:05:11PM -0600, Josh Poimboeuf wrote:
>> On Mon, Dec 12, 2016 at 11:33:54PM +0100, Borislav Petkov wrote:
>> > On Mon, Dec 12, 2016 at 04:11:47PM -0600, Josh Poimboeuf wrote:
>> > > Yes, please.
>> > Attached.
>> Thanks, I was able to recreate. Will take a look tomorrow.
> Figured it out. Your config has CONFIG_PARAVIRT=n, which convinces gcc
> to create the following preamble for x86_64_start_kernel():
> 0000000000000124 <x86_64_start_kernel>:
> 124: 4c 8d 54 24 08 lea 0x8(%rsp),%r10
> 129: 48 83 e4 f0 and $0xfffffffffffffff0,%rsp
> 12d: 41 ff 72 f8 pushq -0x8(%r10)
> 131: 55 push %rbp
> 132: 48 89 e5 mov %rsp,%rbp
> It's an unusual pattern which aligns rsp (though in this case it's
> already aligned) and saves the start_cpu() return address again on the
> stack before storing the frame pointer.
Um, what? I can reproduce it -- I get:
124: 4c 8d 54 24 08 lea 0x8(%rsp),%r10
129: 48 83 e4 f0 and $0xfffffffffffffff0,%rsp
12d: 41 ff 72 f8 pushq -0x8(%r10)
131: 55 push %rbp
132: 48 89 e5 mov %rsp,%rbp
135: 41 57 push %r15
137: 41 56 push %r14
139: 41 55 push %r13
13b: 41 54 push %r12
13d: 41 52 push %r10
13f: 53 push %rbx
140: 48 83 ec 10 sub $0x10,%rsp
and the epilog looks like:
29c: 58 pop %rax
29d: 5a pop %rdx
29e: 5b pop %rbx
29f: 41 5a pop %r10
2a1: 41 5c pop %r12
2a3: 41 5d pop %r13
2a5: 41 5e pop %r14
2a7: 41 5f pop %r15
2a9: 5d pop %rbp
2aa: 49 8d 62 f8 lea -0x8(%r10),%rsp
2ae: c3 retq
This is, I think, *terrible* code. It makes it entirely impossible
for the CPU to look through the retq until the instruction right
before it finishes because there's no way the CPU can tell what rsp is
until the instruction right before the retq. And it's saving and
restoring an entire extra register (r10) instead of just using rbp for
this purpose. *And* the extra copy of the return address seems
totally useless except for unwinding.
This does indeed depend on CONFIG_PARAVIRT, but I'm not seeing what
changes. Presumably something related to what happens in the
I want to file a GCC bug, though. This code sucks.
> The unwinder assumes the last stack frame header is at a certain offset,
> but the above code breaks that assumption. I still need to think about
> the best way to fix it.
Have a dummy written-in-asm top-of-the-stack function? Or recognize
the end by the final saved RBP?