Re: Dealing with holes in CPU address space

From: Chris Packham
Date: Thu Apr 09 2020 - 00:51:20 EST

Next message: Andy Lutomirski: "Re: [PATCH v2] x86/kvm: Disable KVM_ASYNC_PF_SEND_ALWAYS"
Previous message: Sedat Dilek: "Re: [PATCH v2] mac80211: fix race in ieee80211_register_hw()"
In reply to: Florian Fainelli: "Re: Dealing with holes in CPU address space"
Next in thread: Jiaxun Yang: "Re: Dealing with holes in CPU address space"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Wed, 2020-04-08 at 21:03 -0700, Florian Fainelli wrote:
>
> On 4/8/2020 2:33 PM, Chris Packham wrote:
> > On Wed, 2020-04-08 at 15:29 +0800, Jiaxun Yang wrote:
> > > On Wed, 8 Apr 2020 05:14:22 +0000
> > > Chris Packham <Chris.Packham@xxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > > Hi All,
> > > >
> > > > I'm trying to port an old Broadcom MIPS CPU (BCM53003) to a
> > > > shiny
> > > > new
> > > > kernel. I have some old historic source from a long forgotten
> > > > Broadcom
> > > > LDK but I'd prefer to do things the modern way with device-
> > > > trees.
> > > >
> > > > The problem I've been grappling with is trying to open up
> > > > access to
> > > > all of the RAM on the board. It has 512MB of DDR2. The CPU has
> > > > two
> > > > areas where this appears. The first 128MB is from 0 to
> > > > 0x07ffffff
> > > > the
> > > > second area is from 0x88000000 to 0x9fffffff.
> > > >
> > > > SoC peripherals are at 0x18000000 and there is an IO window for
> > > > flash
> > > > at 0x20000000.
> > > >
> > > > The old code has some custom tlb initialisation to deal with
> > > > this
> > > > but
> > > > I figured it should be possible with the following dts snippet.
> > > >
> > > > memory@0 {
> > > > device_type = "memory";
> > > > reg = <0x00000000 0x08000000
> > > > 0x88000000 0x18000000>;
> > > > };
> > > >
> > > > I end up with only 128MB available. This appears to be
> > > > because the default HIGHMEM_START of 0x20000000 stops the rest
> > > > from
> > > > being made available. If I add an override of HIGHMEM_START to
> > > > 0xffffffff I seem to have the full 512MB avaiable but then I
> > > > get a
> > > > kernel panic
> > >
> > > Hi,
> > >
> > > Have you tried to enable CONFIG_HIGHMEM?
> > >
> >
> > I have but that didn't seem to help. As I understand it HIGHMEM is
> > intended for situations when you have more physical RAM that can be
> > addressed (e.g. >4GB on a 32-bit system).
>
> On MIPS you may have to enable HIGHMEM as soon as you run out of
> virtual
> kernel address space to map the entire amount of memory that is
> populated AFAICT. The kernel has a little under 1GB of virtual
> address
> space that can be mapped via the TLB since the first 512MB are
> occupied
> by KSEG0/1.
>

My adventures thus far with HIGHMEM have got as far as

This processor doesn't support highmem. 2490368k highmem ignored

Which I think has something to do with the max_low_pfn and highend_pfn
being different.

> >
> > > >
> > > > CPU 0 Unable to handle kernel paging request at virtual
> > > > address
> > > > 1fc00000, epc == 800167b8, ra == 800e2860
> > > >
> > > > 0x1fc00000 is in the range where the SoC peripherals are so I'm
> > > > thinking that is the problem. But then again that is a virtual
> > > > address
> > > > so maybe it's just a co-incidence.
> > >
> > > 0x1fc00000 should be the Boot ROM's physical address. Probably
> > > you
> > > forgot to convert it into virtual address in your platform code?
> >
> > Yes that's right it's the bootroms PA. I'm not intitentionally
> > doing
> > anything with it but maybe that's the problem.
>
> If you were accessing this as a virtual address then it would be
> either
> via KSEG0/1 and the addresses would be 0x1fc0_0000 + 0x8000_0000 or
> 0x1fc0_0000 + 0xa000_0000 but here it looks like the raw physical
> address is being accessed which suggests the TLB is incorrectly
> programmed somehow.
>
> >
> > >
> > > Check the EPC of exception in vmlinux with addr2line may help.
> > > (Don't
> > > forget to compile your kernel with debuginfo).
> > >
> >
> > The full panic is
> >
> > CPU 0 Unable to handle kernel paging request at virtual address
> > 1fc00000, epc == 800167b8, ra ==
> > 800e2860
> >
> >
> > Oops[#1]:
> >
> >
> >
> > CPU: 0 PID: 0 Comm: swapper Not tainted 5.5.0-at1
> > #46
> >
> >
> > $ 0 : 00000000 00000001 00000001
> > 807e4270
> >
> >
> >
> > $ 4 : 1fc00000 00000000 1fc00f80
> > 00000001
> >
> >
> >
> > $ 8 : 83e52a00 83e52a24 807f628c
> > 8080fa00
> >
> >
> >
> > $12 : ffffffff 00000001 0000000b
> > ffffffff
> >
> >
> >
> > $16 : 83e52a20 80000000 80870000
> > 83e52a00
> >
> >
> >
> > $20 : 8080fdd4 807dde00 00000000
> > 8080f9bc
> >
> >
> >
> > $24 : 8080fb68
> > ffffff7f
> >
> >
> >
> > $28 : 807dc000 807ddd38 83e52a00
> > 800e2860
> >
> >
> >
> > Hi :
> > 00000000
> >
> >
> >
> > Lo :
> > 00000000
> >
> >
> >
> > epc : 800167b8
> > clear_page+0x18/0x128
> >
> >
> >
> > ra : 800e2860
> > get_page_from_freelist+0xa94/0xdd4
> >
> >
> >
> > Status: 11000002 KERNEL
> > EXL
> >
> >
> >
> > Cause : 4080000c (ExcCode
> > 03)
> >
> >
> >
> > BadVA :
> > 1fc00000
> >
> >
> >
> > PrId : 00019749 (MIPS
> > 74Kc)
> >
> >
> >
> > Modules linked
> > in:
> >
> >
> >
> > Process swapper (pid: 0, threadinfo=(ptrval), task=(ptrval),
> > tls=00000000)
> >
> >
> > *HwTLS:
> > e19f7d3a
> >
> >
> >
> > Stack : 807e0000 807dde98 80867cd8 80791814 00000001 80687974
> > 807dde18
> > 00000000
> >
> >
> > 807f5ed0 807f628c 807f628c 8080fdd4 80870000 00000001
> > 807e0000
> > 00000044
> >
> >
> > 0000005c 807dde00 00000000 00000001 80810000 8075a660
> > 80870001
> > 00000000
> >
> >
> > 00000100 00000000 807e0000 00000000 00000000 00000001
> > 80860000
> > 808492b4
> >
> >
> > 87d75608 800e3028 00000100 00000000 00000001 80058c08
> > 80870000
> > 80870000
> >
> >
> > ...
> >
> >
> >
> > Call
> > Trace:
> >
> >
> >
> > [<800167b8>]
> > clear_page+0x18/0x128
> >
> >
> >
> > [<800e2860>]
> > get_page_from_freelist+0xa94/0xdd4
> >
> >
> >
> > [<800e3028>]
> > __alloc_pages_nodemask+0xf4/0xbb8
> >
> >
> >
> > [<800e3b08>]
> > __get_free_pages+0x1c/0x58
> >
> >
> >
> > [<80013430>]
> > setup_zero_pages+0x3c/0xe4
> >
> >
> >
> > [<80826eac>]
> > mem_init+0x40/0x50
> >
> >
> >
> > [<808219c0>]
> > start_kernel+0x250/0x510
> >
> >
> >
> > Code: cc9e0040 cc9e0060 cc9e0080 <ac800000>
> > ac800004 ac800008 ac80000c 24840020 ac80fff0
> >
> > I think this is just the early setup of the pages.
> >
> > > >
> > > > Anyway I'd really appreciate any guidance that anyone could
> > > > provide
> > > > on
> > > > this. Even if it's just "go look at this SoC".
> > > >
> > > > Thanks,
> > > > Chris
> > > >
> > > >
> > >
> > > --
> > > Jiaxun Yang
>
>

Next message: Andy Lutomirski: "Re: [PATCH v2] x86/kvm: Disable KVM_ASYNC_PF_SEND_ALWAYS"
Previous message: Sedat Dilek: "Re: [PATCH v2] mac80211: fix race in ieee80211_register_hw()"
In reply to: Florian Fainelli: "Re: Dealing with holes in CPU address space"
Next in thread: Jiaxun Yang: "Re: Dealing with holes in CPU address space"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]