[PATCH v5 0/3] arm64: hibernate: Resume when hibernate image created on non-boot CPU
From: James Morse
Date: Wed Aug 17 2016 - 08:50:47 EST
Hi all,
These patches allow arm64 to hibernate on any CPU saving the cpu-id in the
arch header, then switch to it during resume using
hibernate_resume_nonboot_cpu_disable().
I hoped to avoid patch 1 by duplicating the logic in the arch code, but
Lorenzo pointed out using cpu_down() indicates that tasks are not frozen,
whereas in reality they are. [0]
Patch 1 lets us specify which cpu disable_nonboot_cpus() should leave
standing.
(All three patches need to be merged together, this series doesn't conflict
with [1].)
Comments welcome,
This series is based on v4.8-rc2 and can be retrieved from:
git://linux-arm.org/linux-jm.git hibernate/cpuN/v5
Changes since v4:
* Added freeze_secondary_cpus().
* Wired up hibernate_resume_nonboot_cpu_disable(), removing macros and
kconfig symbols from previous approaches.
* Added check for sleep_cpu being uninitialised when we come to save the
arch header.
Changes since v3:
* Split series, at the PM/Hibernate patch, merged arm64 patches to remove
dependencies.
* Changed Kconfig symbol name.
* Fixed logic error '<= 0' when testing for an unrecognised CPU.
Changes since v2:
* Split wrong-CPU logic into an earlier patch that just returns an error.
* Changed core code patch to use macros instead of a weak function.
CONFIG_ARCH_HIBERNATION_HEADER now implies an asm/suspend.h header.
* Wording in error messages 'hibernate' not 'suspend'.
Changes since v1:
* Fixed 'Brining' typo.
[v1] http://www.spinics.net/lists/arm-kernel/msg507805.html
[v2] https://www.spinics.net/lists/arm-kernel/msg511654.html
[v3] https://www.spinics.net/lists/arm-kernel/msg514644.html
[v4] https://www.spinics.net/lists/arm-kernel/msg515978.html
[0] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/441305.html
[1] https://www.spinics.net/lists/arm-kernel/msg523933.html
[2] hotplug cpu0, kexec, hibernate, resume
-------------------------%<-------------------------
root@ubuntu:~# echo disk > /sys/power/state
[ 76.768682] PM: Syncing filesystems ... done.
[ 76.774872] Freezing user space processes ... (elapsed 0.001 seconds) done.
[ 76.783253] Double checking all user space processes after OOM killer disable
... (elapsed 0.000 seconds)
[ 76.794214] PM: Preallocating image memory... done (allocated 84746 pages)
[ 80.454878] PM: Allocated 338984 kbytes in 3.65 seconds (92.87 MB/s)
[ 80.461269] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[ 80.491476] PM: freeze of devices complete after 21.037 msecs
[ 80.498053] PM: late freeze of devices complete after 0.789 msecs
[ 80.505563] PM: noirq freeze of devices complete after 1.387 msecs
[ 80.511777] Disabling non-boot CPUs ...
[ 80.540044] CPU1: shutdown
[ 80.542755] psci: CPU1 killed.
[ 80.611931] CPU2: shutdown
[ 80.614646] psci: CPU2 killed.
[ 80.687816] IRQ11 no longer affine to CPU3
[ 80.687831] IRQ20 no longer affine to CPU3
[ 80.687872] CPU3: shutdown
[ 80.698806] psci: CPU3 killed.
[ 80.725600] PM: Creating hibernation image:
[ 80.725600] PM: Need to copy 84054 pages
[ 80.725600] PM: Hibernation image created (84054 pages copied)
[ 80.725609] Enabling non-boot CPUs ...
[ 80.758320] Detected VIPT I-cache on CPU1
[ 80.758381] CPU1: Booted secondary processor [410fd033]
[ 80.758946] CPU1 is up
[ 80.810889] Detected VIPT I-cache on CPU2
[ 80.810932] CPU2: Booted secondary processor [410fd033]
[ 80.811598] CPU2 is up
[ 80.863596] Detected VIPT I-cache on CPU3
[ 80.863640] CPU3: Booted secondary processor [410fd033]
[ 80.864256] CPU3 is up
[ 80.876712] PM: noirq thaw of devices complete after 0.802 msecs
[ 80.883475] PM: early thaw of devices complete after 0.699 msecs
[ 80.947177] PM: thaw of devices complete after 57.665 msecs
[ 80.953222] hibernate: Hibernating on CPU 0 [mpidr:0x101]
[ 80.961099] PM: Using 3 thread(s) for compression.
[ 80.961099] PM: Compressing and saving image data (84219 pages)...
[ 80.972154] PM: Image saving progress: 0%
[ 81.270381] PM: Image saving progress: 10%
[ 81.498242] PM: Image saving progress: 20%
[ 81.789014] PM: Image saving progress: 30%
[ 84.844984] PM: Image saving progress: 40%
[ 87.657013] PM: Image saving progress: 50%
[ 88.933987] PM: Image saving progress: 60%
[ 90.597472] PM: Image saving progress: 70%
[ 92.362916] PM: Image saving progress: 80%
[ 93.622648] PM: Image saving progress: 90%
[ 95.468855] PM: Image saving progress: 100%
[ 99.073785] PM: Image saving done.
[ 99.077240] PM: Wrote 336876 kbytes in 18.10 seconds (18.61 MB/s)
[ 99.092099] PM: S|
[ 99.370393] kvm: exiting hardware virtualization
[ 99.535936] reboot: Restarting system
[ ... ]
[ 6.074235] Freezing user space processes ... (elapsed 0.000 seconds) done.
[ 6.081540] Double checking all user space processes after OOM killer disable
... (elapsed 0.000 seconds)
[ 6.119415] random: fast init done
[ 6.133638] PM: Using 1 thread(s) for decompression.
[ 6.133638] PM: Loading and decompressing image data (84219 pages)...
[ 6.145079] hibernate: Hibernated on CPU 1 [mpidr:0x101]
[ 6.150412] hibernate: Hibernated on a CPU that is offline! Bringing CPU up.
[ 6.198971] Detected VIPT I-cache on CPU1
[ 6.199036] CPU1: Booted secondary processor [410fd033]
[ 7.177507] PM: Image loading progress: 0%
[ 7.816126] PM: Image loading progress: 10%
[ 8.176005] PM: Image loading progress: 20%
[ 8.568329] PM: Image loading progress: 30%
[ 8.991993] PM: Image loading progress: 40%
[ 9.385313] PM: Image loading progress: 50%
[ 9.708479] PM: Image loading progress: 60%
[ 10.082700] PM: Image loading progress: 70%
[ 10.478748] PM: Image loading progress: 80%
[ 10.876730] PM: Image loading progress: 90%
[ 11.275249] PM: Image loading progress: 100%
[ 11.280021] PM: Image loading done.
[ 11.283534] PM: Read 336876 kbytes in 5.13 seconds (65.66 MB/s)
[ 11.316174] PM: quiesce of devices complete after 19.385 msecs
[ 11.322864] PM: late quiesce of devices complete after 0.825 msecs
[ 11.349479] PM: noirq quiesce of devices complete after 20.401 msecs
[ 11.355864] Disabling non-boot CPUs ...
[ 11.372531] IRQ7 no longer affine to CPU0
[ 11.372537] IRQ9 no longer affine to CPU0
[ 11.372544] IRQ11 no longer affine to CPU0
[ 11.372559] IRQ20 no longer affine to CPU0
[ 11.372595] CPU0: shutdown
[ 11.391601] psci: CPU0 killed.
[ 80.725613] Enabling non-boot CPUs ...
[ 80.758314] Detected VIPT I-cache on CPU1
[ 80.758375] CPU1: Booted secondary processor [410fd033]
[ 80.758939] CPU1 is up
[ 80.810877] Detected VIPT I-cache on CPU2
[ 80.810917] CPU2: Booted secondary processor [410fd033]
[ 80.811581] CPU2 is up
[ 80.859588] Detected VIPT I-cache on CPU3
[ 80.859628] CPU3: Booted secondary processor [410fd033]
[ 80.860246] CPU3 is up
[ 80.873337] PM: noirq restore of devices complete after 1.434 msecs
[ 80.880381] PM: early restore of devices complete after 0.701 msecs
[ 81.231471] PM: restore of devices complete after 71.157 msecs
[ 81.238336] Restarting tasks ... done.
root@ubuntu:~#
-------------------------%<-------------------------
James Morse (3):
cpu/hotplug: Allow suspend/resume CPU to be specified
arm64: hibernate: Resume when hibernate image created on non-boot CPU
Revert "arm64: hibernate: Refuse to hibernate if the boot cpu is
offline"
arch/arm64/include/asm/suspend.h | 3 ++
arch/arm64/kernel/hibernate.c | 67 +++++++++++++++++++++++++++-------------
include/linux/cpu.h | 6 +++-
kernel/cpu.c | 9 +++---
4 files changed, 59 insertions(+), 26 deletions(-)
--
2.8.0.rc3