Re: [PATCH] x86/entry/x32: Check top 32 bits of syscall number on the fast path

[Andy Lutomirski]

On Sun, Apr 17, 2016 at 9:50 PM, H. Peter Anvin <hpa@xxxxxxxxx> wrote:
> On 04/17/16 17:47, Ben Hutchings wrote:
>> We've always masked off the top 32 bits when x32 is enabled, but
>> hopefully no-one relies on that.  Now that the slow path is in C, we
>> check all the bits there, regardless of whether x32 is enabled.  Let's
>> make the fast path consistent with it.
>
> We have always masked off the top 32 bits *period*.
>
> We have had some bugs where we haven't, because someone has tried to
> "optimize" the code and they have been quite serious.  The system call
> number is an int, which means the upper 32 bits are undefined on call
> entry: we HAVE to mask them.

I'm reasonably confident that normal kernels (non-x32) have not masked
those bits since before I started hacking on the entry code.

So the type of the syscall nr is a bit confused.  If there was an
installed base of programs that leaved garbage in the high bits, we
would have noticed *years* ago.  On the other hand, the 32-bit ptrace
ABI and the seccomp ABI both think it's 32-bits.

If we were designing the x86_64 ABI and everything around it from
scratch, I'd suggest that that either the high bits must be zero or
that the number actually be 64 bits (which are more or less the same
thing).  That would let us use the high bits for something interesting
in the future.

In practice, we can probably still declare that the thing is a 64-bit
number, given that most kernels in the wild currently fail syscalls
that have the high bits set.

--Andy