Re: [PATCH 3/3] x86/crash: Allocate enough low memory when crashkernel=high

[Baoquan He]

On 06/08/15 at 05:20pm, Yinghai Lu wrote:
> On Mon, Jun 8, 2015 at 12:49 AM, Borislav Petkov <bp@xxxxxxxxx> wrote:
> > From: Joerg Roedel <jroedel@xxxxxxx>
> >
> > When the crash kernel is loaded above 4GiB in memory, the
> > first kernel allocates only 72MiB of low-memory for the DMA
> > requirements of the second kernel. On systems with many
> > devices this is not enough and causes device driver
> > initialization errors and failed crash dumps. Testing by
> > SUSE and Redhat has shown that 256MiB is a good default
> > value for now and the discussion has lead to this value as
> > well. So set this default value to 256MiB to make sure there
> > is enough memory available for DMA.

Hi Yinghai,

> 
> Is the 256MiB too big?
> 
> BTW, the affected system does not support iommu?

I guess the affected system should not support hw iommu or turned off hw
iommu since hw iommu always screw up kdump.

> 
> on my intel test box with 6T and 16 pcie cards does not hit the 72MiB
> limit.

In fact in this case, it doesn't matter how much memory the system has
since most of them is above 4G and only 72M is reserved for dma/swiotlb
in kdump kernel. And it doesn't matter much how many pci/pcie cards the
system has since it matters how greedy those devices expects dma
allocation.

Before Joerg posted this patchset v1, our customer complained about
this too. That means this is not individual case. I think it makes sense
to increase the default low-memory to make kdump succeed, it's
definitely better than letting user get a failed kdump and then know
they need specify more low-memory manually. If user clearly know their
system doesn't need so much low-memory, specifying crashkernel=xx,low
works for them.
> 
> So the affected system is 12T and more cards?

In the case of my bug, it's a system with 12T and many cards.

Thanks
Baoquan
> 
> >
> > Signed-off-by: Joerg Roedel <jroedel@xxxxxxx>
> > Acked-by: Baoquan He <bhe@xxxxxxxxxx>
> > Cc: Baoquan He <bhe@xxxxxxxxxx>
> > Cc: Dave Young <dyoung@xxxxxxxxxx>
> > Cc: "H. Peter Anvin" <hpa@xxxxxxxxx>
> > Cc: Ingo Molnar <mingo@xxxxxxxxxx>
> > Cc: Jörg Rödel <joro@xxxxxxxxxx>
> > Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> > Cc: Vivek Goyal <vgoyal@xxxxxxxxxx>
> > Cc: kexec@xxxxxxxxxxxxxxxxxxx
> > Cc: x86-ml <x86@xxxxxxxxxx>
> > Link: http://lkml.kernel.org/r/1433500202-25531-4-git-send-email-joro@xxxxxxxxxx
> > [ Reflow comment. ]
> > Signed-off-by: Borislav Petkov <bp@xxxxxxx>
> > ---
> >  arch/x86/kernel/setup.c | 13 ++++++++-----
> >  1 file changed, 8 insertions(+), 5 deletions(-)
> >
> > diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> > index d74ac33290ae..5a697e56634c 100644
> > --- a/arch/x86/kernel/setup.c
> > +++ b/arch/x86/kernel/setup.c
> > @@ -531,12 +531,15 @@ static void __init reserve_crashkernel_low(void)
> >         if (ret != 0) {
> >                 /*
> >                  * two parts from lib/swiotlb.c:
> > -                *      swiotlb size: user specified with swiotlb= or default.
> > -                *      swiotlb overflow buffer: now is hardcoded to 32k.
> > -                *              We round it to 8M for other buffers that
> > -                *              may need to stay low too.
> > +                * - swiotlb size: user-specified with swiotlb= or default.
> > +                *
> > +                * -swiotlb overflow buffer: now hardcoded to 32k.
> 
> why do you need to treat them differently ?
> "- swiotlb" and "-swiotlb"
> 
> > +                *
> 
> Why do you need to put extra blank line here? that 8M round up
> is for swiotlb overflow buffer.
> 
> > +                * We round it to 8M for other buffers that may need to stay low
> > +                * too. Also make sure we allocate enough extra low memory so
> > +                * that we don't run out of DMA buffers for 32-bit devices.
> >                  */
> > -               low_size = swiotlb_size_or_default() + (8UL<<20);
> > +               low_size = max(swiotlb_size_or_default() + (8UL<<20), 256UL<<20);
> >                 auto_set = true;
> >         } else {
> >                 /* passed with crashkernel=0,low ? */
> > --
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/