Re: Runtime PM causes oops on next-20151015

[Thierry Reding]

On Thu, Oct 15, 2015 at 09:04:54PM +1100, Michael Ellerman wrote:
> On Thu, 2015-10-15 at 11:22 +0200, Thierry Reding wrote:
> > Hi Rafael, Wonhong,
> > 
> > Todays linux-next breaks rather spectacularly for drivers using runtime
> > PM. The culprit seems to be this commit:
> > 
> > 	commit 7d24068e144adc03b805806645d732cf79488717
> > 	Author: Wonhong Kwon <wonhongkwon@xxxxxxxxx>
> > 	Date:   Tue Oct 6 10:10:20 2015 +0900
> > 
> > 	    PM / hibernate: Move pm_init/pm_disk_init to late_initcall_sync
> > 
> > 	    pm_init is being invoked by core_initcall and hibernate_image_size_init
> > 	    calculates preferred image size (image_size) based on total pages
> > 	    (totalram_pages). This totalram_pages can be modified during various
> > 	    initcall-s phase and this can cause miscalculated image_size.
> > 
> > 	    For example, when CMA is being used, init_cma_reserved_pageblock tries
> > 	    to change the totalram_pages and this job is done during core_initcall.
> > 	    In order words, the totalram_pages doesn't take CMA reserved pages into
> > 	    account when image_size is calculated and it can be too small.
> > 
> > 	    Move pm_init and pm_disk_init to late_initcall_sync so that it happens
> > 	    after all other initcall-s change the totalram_pages.
> > 
> > 	    Reported-by: Sangseok Lee <sangseok.lee@xxxxxxx>
> > 	    Signed-off-by: Wonhong Kwon <wonhong.kwon@xxxxxxx>
> > 	    Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@xxxxxxxxx>
> > 
> 
> I'm seeing one too on powerpc:
> 
> Unable to handle kernel paging request for data at address 0x00000030
> Faulting instruction address: 0xc0000000002e4094
> Oops: Kernel access of bad area, sig: 11 [#1]
> SMP NR_CPUS=2048 NUMA pSeries
> Modules linked in:
> CPU: 5 PID: 1 Comm: swapper/0 Not tainted 4.3.0-rc5-next-20151015-50217-ge2be40d-dirty #37
> task: c0000001fefc0000 ti: c0000001fb000000 task.ti: c0000001fb000000
> NIP: c0000000002e4094 LR: c000000000c79dd4 CTR: c000000000c79d90
> REGS: c0000001fb003900 TRAP: 0300   Not tainted  (4.3.0-rc5-next-20151015-50217-ge2be40d-dirty)
> MSR: 8000000100009032 <SF,EE,ME,IR,DR,RI>  CR: 28000882  XER: 20000000
> CFAR: c0000000000d117c DAR: 0000000000000030 DSISR: 40000000 SOFTE: 1 
> GPR00: c00000000000b074 c0000001fb003b80 c000000000e93800 0000000000000000 
> GPR04: c000000000d15c80 0000000000000000 00000001ff150000 0000000000000026 
> GPR08: c000000000d0f748 c000000000f41a10 c000000000d0f8e8 c000000000abc59d 
> GPR12: c000000000b42400 c00000000fdc1680 c00000000000b960 0000000000000000 
> GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
> GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
> GPR24: 0000000000000000 c000000000de0360 c000000000c45d28 0000000000000000 
> GPR28: c000000000deca80 c000000000d103c0 c0000001fbc905c0 c000000000d103c0 
> NIP [c0000000002e4094] .sysfs_create_file_ns+0x4/0x50
> LR [c000000000c79dd4] .__machine_initcall_pseries_apo_pm_init+0x44/0x60
> Call Trace:
> [c0000001fb003b80] [c0000000004852cc] .kasprintf+0x2c/0x40 (unreliable)
> [c0000001fb003bf0] [c00000000000b074] .do_one_initcall+0xc4/0x250
> [c0000001fb003ce0] [c000000000c64560] .kernel_init_freeable+0x268/0x348
> [c0000001fb003db0] [c00000000000b97c] .kernel_init+0x1c/0x140
> [c0000001fb003e30] [c0000000000095ac] .ret_from_kernel_thread+0x58/0xac
> Instruction dump:
> 419e001c 3ce2ffa8 38e7be60 4bfffea4 3ce2ffa8 38e7bea8 4bfffe9c 3ce2ffa8 
> 38e7bea8 4bfffe8c 60000000 2c230000 <e8630030> 41820028 2fa30000 419e0020 
> ---[ end trace d2a742ce359651dd ]---
> 
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> 
> 
> Which starts at:
> 
>   static int __init apo_pm_init(void)
>   {
>   	return (sysfs_create_file(power_kobj, &auto_poweron_attr.attr));
>   }
>   machine_device_initcall(pseries, apo_pm_init);
> 
> And seems to be caused by power_kobj being NULL:
> 
>   0:mon> d $power_kobj
>   c000000000f41a10 0000000000000000 0000000000000000  |................|

Yes, power_kobj is one of the other objects created by pm_init() and
which, evidently, needs to be available much earlier than late_initcall.
The patch below works for me, though I don't use hibernate on this
machine, so that part is untested.

Thierry

--- >8 ---
diff --git a/kernel/power/main.c b/kernel/power/main.c
index b2dd4d999900..2f66453e3a9b 100644
--- a/kernel/power/main.c
+++ b/kernel/power/main.c
@@ -648,8 +648,6 @@ static int __init pm_init(void)
 	int error = pm_start_workqueue();
 	if (error)
 		return error;
-	hibernate_image_size_init();
-	hibernate_reserved_size_init();
 	power_kobj = kobject_create_and_add("power", NULL);
 	if (!power_kobj)
 		return -ENOMEM;
@@ -661,3 +659,13 @@ static int __init pm_init(void)
 }
 
 core_initcall(pm_init);
+
+static int __init hibernate_init(void)
+{
+	hibernate_image_size_init();
+	hibernate_reserved_size_init();
+
+	return 0;
+}
+
+late_initcall_sync(hibernate_init);

Attachment: signature.asc
Description: PGP signature