[PATCH 2/2] x86/nmi: Fix and optimize the NMI stack check code

From: Steven Rostedt
Date: Thu Mar 09 2017 - 17:44:59 EST

From: "Steven Rostedt (Red Hat)" <rostedt@xxxxxxxxxxx>

Andy Lutomirski reported an off by one in the NMI stack check
for the nested NMI code, where if the stack pointer was one above
the actual stack (stack start + STACK_SIZE) it would trigger a false
positive. This is not that big of a deal because the stack pointer
should never be that. Even if a stack was using the pages just
above the NMI stack, it would require the stack about to overflow
for this to trigger, which is a much bigger bug than this is fixing.

Also, Linus Torvalds pointed out that doing two compares can be
accomplish with a single compare. That is:

("reg" is top of stack we are comparing "stack" to)

cmpq reg, stack
jae label // note, code had one off "ja" instead of "jae"
subq size, reg
cmpq reg, stack
jb label

Is the same as:

subq $1, reg
subq stack, reg
cmpq size, reg
jae label

The subq $1 was added into the leaq by doing:

leaq 5*8+7(%rsp), %rdx

Added more comments as well.

Reported-by: Andy Lutomirski <luto@xxxxxxxxxxxxxx>
Inspired-by: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Signed-off-by: Steven Rostedt (VMware) <rostedt@xxxxxxxxxxx>
arch/x86/entry/entry_64.S | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 3aad759aace2..1e6ca3740762 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -1355,16 +1355,17 @@ ENTRY(nmi)
* if it controls the kernel's RSP. We set DF before we clear
* "NMI executing".
- lea 6*8(%rsp), %rdx
- /* Compare the NMI stack (rdx) with the stack we came from (4*8(%rsp)) */
- cmpq %rdx, 4*8(%rsp)
- /* If the stack pointer is above the NMI stack, this is a normal NMI */
- ja first_nmi
- subq $EXCEPTION_STKSZ, %rdx
- cmpq %rdx, 4*8(%rsp)
- /* If it is below the NMI stack, it is a normal NMI */
- jb first_nmi
+ /* Load address of the top of this stack into rdx */
+ lea 5*8+7(%rsp), %rdx
+ /* Subtract the return stack pointer from it */
+ subq 4*8(%rsp), %rdx
+ /*
+ * If the result is greater or equal to the stack size,
+ * then the return stack was not on this stack.
+ */
+ cmpq $EXCEPTION_STKSZ, %rdx
+ jae first_nmi

/* Ah, it is within the NMI stack. */