Re: [PATCH v5 2/2] RISC-V: Setup initial page tables in two stages

[Alistair Francis]

On Thu, 2019-08-15 at 12:07 -0700, David Abdurachmanov wrote:
> On Thu, Aug 15, 2019 at 11:57 AM Alistair Francis
> <Alistair.Francis@xxxxxxx> wrote:
> > On Wed, 2019-07-10 at 17:05 -0700, Paul Walmsley wrote:
> > > On Fri, 7 Jun 2019, Anup Patel wrote:
> > > 
> > > > Currently, the setup_vm() does initial page table setup in one-
> > > > shot
> > > > very early before enabling MMU. Due to this, the setup_vm() has
> > > > to
> > > > map
> > > > all possible kernel virtual addresses since it does not know
> > > > size
> > > > and
> > > > location of RAM. This means we have kernel mappings for non-
> > > > existent
> > > > RAM and any buggy driver (or kernel) code doing out-of-bound
> > > > access
> > > > to RAM will not fault and cause underterministic behaviour.
> > > > 
> > > > Further, the setup_vm() creates PMD mappings (i.e. 2M mappings)
> > > > for
> > > > RV64 systems. This means for PAGE_OFFSET=0xffffffe000000000
> > > > (i.e.
> > > > MAXPHYSMEM_128GB=y), the setup_vm() will require 129 pages
> > > > (i.e.
> > > > 516 KB) of memory for initial page tables which is never freed.
> > > > The
> > > > memory required for initial page tables will further increase
> > > > if
> > > > we chose a lower value of PAGE_OFFSET (e.g. 0xffffff0000000000)
> > > > 
> > > > This patch implements two-staged initial page table setup, as
> > > > follows:
> > > > 1. Early (i.e. setup_vm()): This stage maps kernel image and
> > > > DTB in
> > > > a early page table (i.e. early_pg_dir). The early_pg_dir will
> > > > be
> > > > used
> > > > only by boot HART so it can be freed as-part of init memory
> > > > free-
> > > > up.
> > > > 2. Final (i.e. setup_vm_final()): This stage maps all possible
> > > > RAM
> > > > banks in the final page table (i.e. swapper_pg_dir). The boot
> > > > HART
> > > > will start using swapper_pg_dir at the end of setup_vm_final().
> > > > All
> > > > non-boot HARTs directly use the swapper_pg_dir created by boot
> > > > HART.
> > > > 
> > > > We have following advantages with this new approach:
> > > > 1. Kernel mappings for non-existent RAM don't exists anymore.
> > > > 2. Memory consumed by initial page tables is now indpendent of
> > > > the
> > > > chosen PAGE_OFFSET.
> > > > 3. Memory consumed by initial page tables on RV64 system is 2
> > > > pages
> > > > (i.e. 8 KB) which has significantly reduced and these pages
> > > > will be
> > > > freed as-part of the init memory free-up.
> > > > 
> > > > The patch also provides a foundation for implementing strict
> > > > kernel
> > > > mappings where we protect kernel text and rodata using PTE
> > > > permissions.
> > > > 
> > > > Suggested-by: Mike Rapoport <rppt@xxxxxxxxxxxxx>
> > > > Signed-off-by: Anup Patel <anup.patel@xxxxxxx>
> > > 
> > > Thanks, updated to apply and to fix a checkpatch warning, and
> > > queued.
> > > 
> > > This may not make it in for v5.3-rc1; if not, we'll submit it
> > > later.
> > 
> > I'm seeing this failure on RV32 which I bisected to this patch:
> > 
> > [    1.820461] systemd[1]: systemd 242-19-gdb2e367+ running in
> > system
> > mode. (-PAM -AUDIT -SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP
> > -LIBCRYPTSETUP -GCRYPT -GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID
> > -ELFUTILS
> > +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=hybrid)
> > [    1.824320] Unable to handle kernel paging request at virtual
> > address 9ff00c15
> > [    1.824973] Oops [#1]
> > [    1.825162] Modules linked in:
> > [    1.825536] CPU: 0 PID: 1 Comm: systemd Not tainted 5.2.0-rc7 #1
> > [    1.826039] sepc: c05c3c78 ra : c04b5a74 sp : df047ce0
> > [    1.826514]  gp : c07a1038 tp : df04c000 t0 : 000000fc
> > [    1.826919]  t1 : 00000002 t2 : 000003ef s0 : df047cf0
> > [    1.827322]  s1 : df7090f8 a0 : 9ff00c15 a1 : c072166c
> > [    1.827723]  a2 : 00000000 a3 : 00000001 a4 : 00000001
> > [    1.828104]  a5 : df6f8138 a6 : 0000002f a7 : de62a000
> > [    1.828534]  s2 : c072166c s3 : 00000000 s4 : 00000000
> > [    1.828931]  s5 : c07a2000 s6 : 00400cc0 s7 : 00000400
> > [    1.829319]  s8 : de491018 s9 : 00000000 s10: fffff000
> > [    1.829702]  s11: de491030 t3 : de62b000 t4 : 00000000
> > [    1.830090]  t5 : 00000000 t6 : 00000080
> > [    1.830392] sstatus: 00000100 sbadaddr: 9ff00c15 scause:
> > 0000000d
> > [    1.831616] ---[ end trace 49a926a1a5300c00 ]---
> > [    1.835776] Kernel panic - not syncing: Attempted to kill init!
> > exitcode=0x0000000b
> > [    1.836575] ---[ end Kernel panic - not syncing: Attempted to
> > kill
> > init! exitcode=0x0000000b ]---
> > 
> > Does anyone else see this?
> > 
> > A simple revert of this patch on 5.3-rc4 fixes the issue for me.
> 
> Yes, I do see those in Fedora/RISCV build farm every morning, but
> with
> riscv64 and 5.2.0-rc7 kernel.
> 
> You also seem to run 5.2.0-rc7 kernel.

That is just a copy error as I copied the log from my bisect to paste
it into my commit. I can reproduce this on 5.3-rc4 as well.

Alistair

> 
> fedora-riscv-4 login: [178876.406122] Unable to handle kernel paging
> request at virtual address 0000000000012a28
> fedora-riscv-7 login: [17983.074847] Unable to handle kernel paging
> request at virtual address 0fffffdff5e14700
> 
> david