[PATCH] x86/uaccess: Implement get_kernel()

[Ingo Molnar]

* Ingo Molnar <mingo@xxxxxxxxxx> wrote:

> * Ingo Molnar <mingo@xxxxxxxxxx> wrote:
> 
> > Yeah, so what I missed here are those nops: placeholders for the 
> > STAC/CLAC instructions on x86... and this is what Linus mentioned 
> > about the clac() overhead.
> > 
> > But this could be solved I think: by adding a 
> > copy_from_kernel_inatomic() primitive which simply leaves out the 
> > STAC/CLAC sequence: as these are always guaranteed to be kernel 
> > addresses, the SMAP fault should not be generated.
> 
> So the first step would be to introduce a generic 
> __copy_from_kernel_inatomic() primitive as attached below.
> 
> The next patch will implement efficient __copy_from_kernel_inatomic() 
> for x86.


The patch below does that. Note, for simplicity I've changed the 
interface to 'get_kernel()' (will propagate this through the other 
patches as well).

Minimally boot tested. owner-spinning looks sane now:

0000000000000030 <mutex_spin_on_owner.isra.4>:
  30:	48 3b 37             	cmp    (%rdi),%rsi
  33:	55                   	push   %rbp
  34:	48 89 e5             	mov    %rsp,%rbp
  37:	75 4f                	jne    88 <mutex_spin_on_owner.isra.4+0x58>
  39:	48 8d 56 28          	lea    0x28(%rsi),%rdx
  3d:	8b 46 28             	mov    0x28(%rsi),%eax
  40:	85 c0                	test   %eax,%eax
  42:	74 3c                	je     80 <mutex_spin_on_owner.isra.4+0x50>
  44:	65 48 8b 04 25 00 00 	mov    %gs:0x0,%rax
  4b:	00 00 
  4d:	48 8b 80 10 c0 ff ff 	mov    -0x3ff0(%rax),%rax
  54:	a8 08                	test   $0x8,%al
  56:	75 28                	jne    80 <mutex_spin_on_owner.isra.4+0x50>
  58:	65 48 8b 0c 25 00 00 	mov    %gs:0x0,%rcx
  5f:	00 00 
  61:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)
  68:	f3 90                	pause  
  6a:	48 3b 37             	cmp    (%rdi),%rsi
  6d:	75 19                	jne    88 <mutex_spin_on_owner.isra.4+0x58>
  6f:	8b 02                	mov    (%rdx),%eax
  71:	85 c0                	test   %eax,%eax
  73:	74 0b                	je     80 <mutex_spin_on_owner.isra.4+0x50>
  75:	48 8b 81 10 c0 ff ff 	mov    -0x3ff0(%rcx),%rax
  7c:	a8 08                	test   $0x8,%al
  7e:	74 e8                	je     68 <mutex_spin_on_owner.isra.4+0x38>
  80:	31 c0                	xor    %eax,%eax
  82:	5d                   	pop    %rbp
  83:	c3                   	retq   
  84:	0f 1f 40 00          	nopl   0x0(%rax)
  88:	b8 01 00 00 00       	mov    $0x1,%eax
  8d:	5d                   	pop    %rbp
  8e:	c3                   	retq   
  8f:	90                   	nop

although the double unrolled need_resched() check looks silly:

  4d:	48 8b 80 10 c0 ff ff 	mov    -0x3ff0(%rax),%rax
  54:	a8 08                	test   $0x8,%al

  75:	48 8b 81 10 c0 ff ff 	mov    -0x3ff0(%rcx),%rax
  7c:	a8 08                	test   $0x8,%al

Thanks,

	Ingo

=============================>