Re: [PATCH v5 04/18] sparc32: mm: Reduce allocation size for PMD and PTE tables

[Mike Rapoport]

On Thu, May 21, 2020 at 04:02:11PM -0700, Guenter Roeck wrote:
> On 5/20/20 12:51 PM, Mike Rapoport wrote:
> > On Wed, May 20, 2020 at 12:03:31PM -0700, Guenter Roeck wrote:
> >> On 5/20/20 10:03 AM, Mike Rapoport wrote:
> >>> On Mon, May 18, 2020 at 09:37:15AM +0100, Will Deacon wrote:
> >>>> On Sat, May 16, 2020 at 05:07:50PM -0700, Guenter Roeck wrote:
> >>>>> On Sat, May 16, 2020 at 05:00:50PM -0700, Guenter Roeck wrote:
> >>>>>> On Mon, May 11, 2020 at 09:41:36PM +0100, Will Deacon wrote:
> >>>>>>> Now that the page table allocator can free page table allocations
> >>>>>>> smaller than PAGE_SIZE, reduce the size of the PMD and PTE allocations
> >>>>>>> to avoid needlessly wasting memory.
> >>>>>>>
> >>>>>>> Cc: "David S. Miller" <davem@xxxxxxxxxxxxx>
> >>>>>>> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> >>>>>>> Signed-off-by: Will Deacon <will@xxxxxxxxxx>
> >>>>>>
> >>>>>> Something in the sparc32 patches in linux-next causes all my sparc32 emulations
> >>>>>> to crash. bisect points to this patch, but reverting it doesn't help, and neither
> >>>>>> does reverting the rest of the series.
> >>>>>>
> >>>>> Actually, turns out I see the same pattern (lots of scheduling while atomic
> >>>>> followed by 'killing interrupt handler' in cryptomgr_test) with several
> >>>>> powerpc boot tests.  I am currently bisecting those crashes. I'll report
> >>>>> the results here as well as soon as I have it.
> >>>>
> >>>> FWIW, I retested my sparc32 patches with PREEMPT=y and I don't see any
> >>>> issues. However, linux-next is a different story, where I don't get very far
> >>>> at all:
> >>>>
> >>>> BUG: Bad page state in process swapper  pfn:005b4
> >>
> >> With above patch applied on top of Ira's patch, I get:
> >>
> >> BUG: spinlock recursion on CPU#0, S01syslogd/139
> >>  lock: 0xf5448350, .magic: dead4ead, .owner: S01syslogd/139, .owner_cpu: 0
> >> CPU: 0 PID: 139 Comm: S01syslogd Not tainted 5.7.0-rc6-next-20200518-00002-gb178d2d56f29-dirty #1
> >> [f0067a64 :
> >> do_raw_spin_lock+0xa8/0xd8 ]
> >> [f00d5034 :
> >> copy_page_range+0x328/0x804 ]
> >> [f0025be4 :
> >> dup_mm+0x334/0x434 ]
> >> [f0027124 :
> >> copy_process+0x1224/0x12b0 ]
> >> [f0027344 :
> >> _do_fork+0x54/0x30c ]
> >> [f0027670 :
> >> do_fork+0x5c/0x6c ]
> >> [f000de44 :
> >> sparc_do_fork+0x18/0x38 ]
> >> [f000b7f4 :
> >> do_syscall+0x34/0x40 ]
> >> [5010cd4c :
> >> 0x5010cd4c ]
> >>
> >> Looks like yet another problem.
> > 
> > I've checked the patch above on top of the mmots which already has Ira's
> > patches and it booted fine. I've used sparc32_defconfig to build the
> > kernel and qemu-system-sparc with default machine and CPU. 
> > 
> 
> Try sparc32_defconfig+SMP.
 
I see a differernt problem, but this could be related:

INIT: version 2.86 booting
rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
	(detected by 0, t=5252 jiffies, g=-935, q=3)
rcu: All QSes seen, last rcu_sched kthread activity 5252 (-68674--73926), jiffies_till_next_fqs=1, root ->qsmask 0x0
rcu: rcu_sched kthread starved for 5252 jiffies! g-935 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
rcu_sched       R  running task        0    10      2 0x00000000

I'm running a bit old debian [1] with qemu-img-sparc.

My bisect pointed at commit 8c8f3156dd40 ("sparc32: mm: Reduce
allocation size for PMD and PTE tables"). The commit ID is valid for
next-20200522.

If I revert this commit and fixup the page table initialization [2] I've
broken, the build with CONFIG_SMP=n works fine, but the build with
CONFIG_SMP=y does not work even if I add nosmp to the kernel command
line. 

[1] https://people.debian.org/~aurel32/qemu/sparc/debian_etch_sparc_small.qcow2
[2] sparc32 meminit fixup:

diff --git a/arch/sparc/mm/init_32.c b/arch/sparc/mm/init_32.c
index e45160839f79..eb2946b1df8a 100644
--- a/arch/sparc/mm/init_32.c
+++ b/arch/sparc/mm/init_32.c
@@ -192,6 +192,7 @@ unsigned long __init bootmem_init(unsigned long *pages_avail)
 	/* Reserve the kernel text/data/bss. */
 	size = (start_pfn << PAGE_SHIFT) - phys_base;
 	memblock_reserve(phys_base, size);
+	memblock_add(phys_base, size);
 
 	size = memblock_phys_mem_size() - memblock_reserved_size();
 	*pages_avail = (size >> PAGE_SHIFT) - high_pages;
diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c
index 75b56bdd38ef..6cb1ea2d2b5c 100644
--- a/arch/sparc/mm/srmmu.c
+++ b/arch/sparc/mm/srmmu.c
@@ -304,7 +304,7 @@ static void __init srmmu_nocache_init(void)
 		pgd = pgd_offset_k(vaddr);
 		p4d = p4d_offset(__nocache_fix(pgd), vaddr);
 		pud = pud_offset(__nocache_fix(p4d), vaddr);
-		pmd = pmd_offset(__nocache_fix(pud), vaddr);
+		pmd = pmd_offset(__nocache_fix(pgd), vaddr);
 		pte = pte_offset_kernel(__nocache_fix(pmd), vaddr);
 
 		pteval = ((paddr >> 4) | SRMMU_ET_PTE | SRMMU_PRIV);

> Guenter

-- 
Sincerely yours,
Mike.