Re: [PATCH RFC 0/2] KVM: s390: avoid having to enable vm.alloc_pgste

[Martin Schwidefsky]

On Fri, 2 Jun 2017 15:20:33 +0200
Christian Borntraeger <borntraeger@xxxxxxxxxx> wrote:

> On 06/02/2017 12:53 PM, Martin Schwidefsky wrote:
> > On Fri, 2 Jun 2017 12:19:19 +0200
> > Christian Borntraeger <borntraeger@xxxxxxxxxx> wrote:
> >   
> >> On 06/02/2017 11:46 AM, Martin Schwidefsky wrote:  
> >>> On Fri, 2 Jun 2017 09:02:10 +0200
> >>> Heiko Carstens <heiko.carstens@xxxxxxxxxx> wrote:
> >>>     
> >>>> On Thu, Jun 01, 2017 at 12:46:51PM +0200, Martin Schwidefsky wrote:    
> >>>>>> Unfortunately, converting all page tables to 4k pgste page tables is
> >>>>>> not possible without provoking various race conditions.      
> >>>>>
> >>>>> That is one approach we tried and was found to be buggy. The point is that
> >>>>> you are not allowed to reallocate a page table while a VMA exists that is
> >>>>> in the address range of that page table.
> >>>>>
> >>>>> Another approach we tried is to use an ELF flag on the qemu executable.
> >>>>> That does not work either because fs/exec.c allocates and populates the
> >>>>> new mm struct for the argument pages before fs/binfmt_elf.c comes into
> >>>>> play.      
> >>>>
> >>>> How about if you would fail the system call within arch_check_elf() if you
> >>>> detect that the binary requires pgstes (as indicated by elf flags) and then
> >>>> restart the system call?
> >>>>
> >>>> That is: arch_check_elf() e.g. would set a thread flag that future mm's
> >>>> should be allocated with pgstes. Then do_execve() would cleanup everything
> >>>> and return to entry.S. Upon return to userspace we detect this condition
> >>>> and simply restart the system call, similar to signals vs -ERESTARTSYS.
> >>>>
> >>>> That would make do_execve() cleanup everything and upon reentering it would
> >>>> allocate an mm with the pgste flag set.
> >>>>
> >>>> Maybe this is a bit over-simplified, but might work.    
> >>>
> >>> This is not over-simplified at all, that does work:    
> >>
> >>
> >> Nice. Next question is how to integrate that elf flag into the qemu
> >> build environment.  
> > 
> > So far I use a small C program to set the flag:
> > 
> > #include <elf.h>
> > #include <fcntl.h>
> > #include <stdio.h>
> > #include <stdlib.h>
> > 
> > #define ERREXIT(...)                    \
> > do {                                    \
> >         fprintf(stderr, __VA_ARGS__);   \
> >         exit(-1);                       \
> > } while (0)
> > 
> > int main(int argc, char *argv[])
> > {
> >         Elf64_Ehdr ehdr;
> >         int fd, rc;
> > 
> >         if (argc != 2)
> >                 ERREXIT("Usage: %s <elf-file>\n", argv[0]);
> > 
> >         fd = open(argv[1], O_RDWR);
> >         if (fd < 0)
> >                 ERREXIT("Unable to open file %s\n", argv[1]);
> > 
> >         if (pread(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr) ||
> >             memcmp(&ehdr.e_ident, ELFMAG, SELFMAG) != 0 ||
> >                 ehdr.e_ident[EI_CLASS] != ELFCLASS64 ||
> >             ehdr.e_machine != EM_S390)
> >                 ERREXIT("Invalid ELF file %s\n", argv[1]);
> > 
> >         ehdr.e_flags |= 0x00000002;
> > 
> >         if (pwrite(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr))
> >                 ERREXIT("Write to of file %s failed\n", argv[1]);
> > 
> >         close(fd);
> >         return 0;
> > }
> >   
> 
> Thanks for that. I assume this is mostly for testing and we want to have 
> toolchain support for that. Otherwise things like  build-id (the sha variant) 
> might be wrong, no?

Andreas Krebbel suggested to use a ELF segment type to mark the qemu binary,
similar to GNU_STACK. For the kernel part of this solution see below, now
we just need toolchain support to generate a qemu executable with the
PT_S390_REQUEST_PGSTE segment. The -z linker option comes to mind, e.g.
"-z request-pgste"

--