Re: [v2 0/5] arm64: allow to reserve memory for normal kexec kernel
From: Pavel Tatashin
Date: Wed Jul 10 2019 - 11:58:19 EST
On Wed, Jul 10, 2019 at 11:28 AM Matthias Brugger
<matthias.bgg@xxxxxxxxx> wrote:
>
>
>
> On 09/07/2019 20:20, Pavel Tatashin wrote:
> > Changelog
> > v1 - v2
> > - No changes to patches, addressed suggestion from James Morse
> > to add "arm64" tag to cover letter.
> > - Improved cover letter information based on discussion.
> >
> > Currently, it is only allowed to reserve memory for crash kernel, because
> > it is a requirement in order to be able to boot into crash kernel without
> > touching memory of crashed kernel is to have memory reserved.
> >
> > The second benefit for having memory reserved for kexec kernel is
> > that it does not require a relocation after segments are loaded into
> > memory.
> >
> > If kexec functionality is used for a fast system update, with a minimal
> > downtime, the relocation of kernel + initramfs might take a significant
> > portion of reboot.
> >
> > In fact, on the machine that we are using, that has ARM64 processor
> > it takes 0.35s to relocate during kexec, thus taking 52% of kernel reboot
> > time:
> >
> > kernel shutdown 0.03s
> > relocation 0.35s
> > kernel startup 0.29s
> >
> > Image: 13M and initramfs is 24M. If initramfs increases, the relocation
> > time increases proportionally.
> >
> > While, it is possible to add 'kexeckernel=' parameters support to other
> > architectures by modifying reserve_crashkernel(), in this series this is
> > done for arm64 only.
> >
>
> I wonder if we couldn't use the crashkernel reserved memory area for that and
> just add logic to kexec-tools to pass to the kernel a flag (a new magic reboot
> number?) to use the crashkernel memory for that?
> The kernel would then unload the crash/capture system in the reserved memory
> area and reuse the latter for kexec.
> This would also enable the feature for all architectures.
I decided to take another route: enable MMU during kernel relocation
on ARM64. This will eliminate the problem that I am experiencing with
slow relocation.
Pasha
>
> Regards,
> Matthias
>
> > The reason it is so slow on arm64 to relocate kernel is because the code
> > that does relocation does this with MMU disabled, and thus D-Cache and
> > I-Cache must also be disabled.
> >
> > Alternative solution is more complicated: Setup a temporary page table
> > for relocation_routine and also for code from cpu_soft_restart. Perform
> > relocation with MMU enabled, do cpu_soft_restart where MMU and caching
> > are disabled, jump to purgatory. A similar approach was suggested for
> > purgatory and was rejected due to making purgatory too complicated.
> > On, the other hand hibernate does something similar already, but there
> > MMU never needs to be disabled, and also by the time machine_kexec()
> > is called, allocator is not available, as we can't fail to do reboot,
> > so page table must be pre-allocated during kernel load time.
> >
> > Note: the above time is relocation time only. Purgatory usually also
> > computes checksum, but that is skipped, because --no-check is used when
> > kernel image is loaded via kexec.
> >
> > Pavel Tatashin (5):
> > kexec: quiet down kexec reboot
> > kexec: add resource for normal kexec region
> > kexec: export common crashkernel/kexeckernel parser
> > kexec: use reserved memory for normal kexec reboot
> > arm64, kexec: reserve kexeckernel region
> >
> > .../admin-guide/kernel-parameters.txt | 7 ++
> > arch/arm64/kernel/setup.c | 5 ++
> > arch/arm64/mm/init.c | 83 ++++++++++++-------
> > include/linux/crash_core.h | 6 ++
> > include/linux/ioport.h | 1 +
> > include/linux/kexec.h | 6 +-
> > kernel/crash_core.c | 27 +++---
> > kernel/kexec_core.c | 50 +++++++----
> > 8 files changed, 127 insertions(+), 58 deletions(-)
> >