Re: [PATCH 3/3] MIPS: Reduce possibility of kernel panic under CONFIG_SWIOTLB

[Tiezhu Yang]

On 04/21/2020 05:35 PM, Jiaxun Yang wrote:
> On Tue, 21 Apr 2020 17:04:29 +0800
> Tiezhu Yang <yangtiezhu@xxxxxxxxxxx> wrote:
>
> > In the current code, if CONFIG_SWIOTLB is set, when failed to get IO
> > TLB memory from the low pages by plat_swiotlb_setup(), it may lead to
> > the boot process failed with kernel panic.
> Hi Tiezhu,
>
> Thanks for you patch.
>
> Firstly, your commit message should be more straight forward. Please
> describe what you have changed (e.g. MIPS: Set memblock bottom up)
> instead of what you solved.

HI Jiaxun,

Thanks for your suggestion.

I will send a v2 patch used with the following subject:
"MIPS: Make sparse_init() using top-down allocation"

> > (1) On the Loongson and SiByte platform
> > arch/mips/loongson64/dma.c
> > arch/mips/sibyte/common/dma.c
> > void __init plat_swiotlb_setup(void)
> > {
> > 	swiotlb_init(1);
> > }
> >
> > kernel/dma/swiotlb.c
> > void  __init
> > swiotlb_init(int verbose)
> > {
> > ...
> > 	vstart = memblock_alloc_low(PAGE_ALIGN(bytes), PAGE_SIZE);
> > 	if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs,
> > verbose)) return;
> > ...
> > 	pr_warn("Cannot allocate buffer");
> > 	no_iotlb_memory = true;
> > }
> >
> > phys_addr_t swiotlb_tbl_map_single()
> > {
> > ...
> > 	if (no_iotlb_memory)
> > 		panic("Can not allocate SWIOTLB buffer earlier ...");
> > ...
> > }
> >
> > (2) On the Cavium OCTEON platform
> > arch/mips/cavium-octeon/dma-octeon.c
> > void __init plat_swiotlb_setup(void)
> > {
> > ...
> > 	octeon_swiotlb = memblock_alloc_low(swiotlbsize, PAGE_SIZE);
> > 	if (!octeon_swiotlb)
> > 		panic("%s: Failed to allocate %zu bytes align=%lx\n",
> > 		      __func__, swiotlbsize, PAGE_SIZE);
> > ...
> > }
> >
> > Because IO_TLB_DEFAULT_SIZE is 64M, if the rest size of low memory is
> > less than 64M when call plat_swiotlb_setup(), we can easily reproduce
> > the panic case.
> >
> > In order to reduce the possibility of kernel panic when failed to get
> > IO TLB memory under CONFIG_SWIOTLB, it is better to allocate low
> > memory as small as possible before plat_swiotlb_setup(), so make
> > sparse_init() using top-down allocation.
> AFAIK there are some reasons that we set it to bottom_up.
> On some platforms, bootloader won't place cmdline & devicetree into
> reserved memory but place them just after kernel in memory. That means
> if you set it as bottom up, then early allocate memory might collide
> with these boot arguments.
>
> I'm not even sure if it works fine on Loongson with early PMON.
>
> I had met that issue before, the solution for me is to reduce SWIOTLB
> size.
>
> > Reported-by: Juxin Gao <gaojuxin@xxxxxxxxxxx>
> > Co-developed-by: Juxin Gao <gaojuxin@xxxxxxxxxxx>
> > Signed-off-by: Juxin Gao <gaojuxin@xxxxxxxxxxx>
> > Signed-off-by: Tiezhu Yang <yangtiezhu@xxxxxxxxxxx>
> > ---
> >   arch/mips/kernel/setup.c | 10 ++++++++++
> >   1 file changed, 10 insertions(+)
> >
> > diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
> > index 5481a0c..8db533c 100644
> > --- a/arch/mips/kernel/setup.c
> > +++ b/arch/mips/kernel/setup.c
> > @@ -700,7 +700,17 @@ static void __init arch_mem_init(char
> > **cmdline_p) memblock_reserve(crashk_res.start,
> > resource_size(&crashk_res)); #endif
> >   	device_tree_init();
> > +
> > +	/*
> > +	 * In order to reduce the possibility of kernel panic when
> > failed to
> > +	 * get IO TLB memory under CONFIG_SWIOTLB, it is better to
> > allocate
> > +	 * low memory as small as possible before
> > plat_swiotlb_setup(), so
> > +	 * make sparse_init() using top-down allocation.
> > +	 */
> > +	memblock_set_bottom_up(false);
> >   	sparse_init();
> > +	memblock_set_bottom_up(true);
> > +
> >   	plat_swiotlb_setup();
> >   
> >   	dma_contiguous_reserve(PFN_PHYS(max_low_pfn));
> --
> Jiaxun Yang