Re: [RFC PATCH] drivers core: cpu: add hotplug callback to update cpu_dev state to resumed

[Sudeep Holla]

On Thu, Jan 31, 2019 at 04:05:59PM +0000, Sudeep Holla wrote:
> On Thu, Jan 31, 2019 at 12:48:49AM +0100, Rafael J. Wysocki wrote:
> > On Friday, January 25, 2019 4:09:06 PM CET Sudeep Holla wrote:
> > > The sysfs for the cpu caches are managed by adding devices with cpu
> > > as the parent in cpu_device_create() when secondary cpu is brought
> > > onlin. Generally when the secondary CPUs are hotplugged back is as part
> > > of resume from suspend-to-ram, we call cpu_device_create() from the cpu
> > > hotplug state machine while the cpu device associated with that CPU is
> > > not yet ready to be resumed as the device_resume() call happens bit later.
> > > It's not really needed to set the flag is_prepared for cpu devices are
> > > they are mostly pseudo device and hotplug framework deals with state
> > > machine and not managed through the cpu device.
> > >
> > > This often results in annoying warning when resuming:
> > > Enabling non-boot CPUs ...
> > > CPU1: Booted secondary processor
> > >  cache: parent cpu1 should not be sleeping
> > > CPU1 is up
> > > CPU2: Booted secondary processor
> > >  cache: parent cpu2 should not be sleeping
> > > CPU2 is up
> > > .... and so on.
> > >
> > > Just fix the warning by updating the device state quite early.
> > >
> > > Cc: "Rafael J. Wysocki" <rjw@xxxxxxxxxxxxx>
> > > Reported-by: Jisheng Zhang <jszhang@xxxxxxxxxxx>
> > > Reported-by: Steve Longerbeam <slongerbeam@xxxxxxxxx>
> > > Reported-by: Eugeniu Rosca <erosca@xxxxxxxxxxxxxx>
> > > Signed-off-by: Sudeep Holla <sudeep.holla@xxxxxxx>
> > > ---
> > >  drivers/base/cpu.c         | 20 +++++++++++++++++++-
> > >  include/linux/cpuhotplug.h |  1 +
> > >  2 files changed, 20 insertions(+), 1 deletion(-)
> > >
> > > Hi Rafael,
> > >
> > > This is getting reported for quite some time. Let me know if you have
> > > better solution to fix this harmless yet annoying warnings during system
> > > resume.
> >
> > I'd rather have a flag in struct dev_pm_info that will cause the message to
> > be suppressed if set.
> >
> > It could be used for other purposes too then in princple (like skipping the
> > creation of empty "power" attr groups in sysfs for devices that don't
> > need them etc.).
> >
> Thanks for the suggestion. I did quick hack and came up with something
> below. I wanted to run through you once before I materialise it into
> a formal patch to check if I understood your suggestion correctly.
> We can move no_pm_required outside dev_pm_info struct and rename with
> any better names.
>

Sorry for the nag, since the title has RFC, thought there are chances of
this getting lost. Let me know if the below idea aligns with your suggestion ?

Regards,
Sudeep
> -->8
> 
> diff --git i/drivers/base/cpu.c w/drivers/base/cpu.c
> index eb9443d5bae1..b61f9772ed33 100644
> --- i/drivers/base/cpu.c
> +++ w/drivers/base/cpu.c
> @@ -379,6 +379,7 @@ int register_cpu(struct cpu *cpu, int num)
>  	cpu->dev.bus->uevent = cpu_uevent;
>  #endif
>  	cpu->dev.groups = common_cpu_attr_groups;
> +	cpu->dev.power.no_pm_required = true;
>  	if (cpu->hotpluggable)
>  		cpu->dev.groups = hotplugable_cpu_attr_groups;
>  	error = device_register(&cpu->dev);
> @@ -427,6 +428,7 @@ __cpu_device_create(struct device *parent, void *drvdata,
>  	dev->parent = parent;
>  	dev->groups = groups;
>  	dev->release = device_create_release;
> +	dev->power.no_pm_required = true;
>  	dev_set_drvdata(dev, drvdata);
>  
>  	retval = kobject_set_name_vargs(&dev->kobj, fmt, args);
> diff --git i/drivers/base/power/main.c w/drivers/base/power/main.c
> index 0992e67e862b..ed1b133f73db 100644
> --- i/drivers/base/power/main.c
> +++ w/drivers/base/power/main.c
> @@ -124,6 +124,8 @@ void device_pm_unlock(void)
>   */
>  void device_pm_add(struct device *dev)
>  {
> +	if (dev->power.no_pm_required)
> +		return;
>  	pr_debug("PM: Adding info for %s:%s\n",
>  		 dev->bus ? dev->bus->name : "No Bus", dev_name(dev));
>  	device_pm_check_callbacks(dev);
> @@ -142,6 +144,8 @@ void device_pm_add(struct device *dev)
>   */
>  void device_pm_remove(struct device *dev)
>  {
> +	if (dev->power.no_pm_required)
> +		return;
>  	pr_debug("PM: Removing info for %s:%s\n",
>  		 dev->bus ? dev->bus->name : "No Bus", dev_name(dev));
>  	complete_all(&dev->power.completion);
> diff --git i/drivers/base/power/sysfs.c w/drivers/base/power/sysfs.c
> index d713738ce796..54c1bfec396e 100644
> --- i/drivers/base/power/sysfs.c
> +++ w/drivers/base/power/sysfs.c
> @@ -648,6 +648,9 @@ int dpm_sysfs_add(struct device *dev)
>  {
>  	int rc;
>  
> +	if (dev->power.no_pm_required)
> +		return 0;
> +
>  	rc = sysfs_create_group(&dev->kobj, &pm_attr_group);
>  	if (rc)
>  		return rc;
> diff --git i/include/linux/pm.h w/include/linux/pm.h
> index 0bd9de116826..300ab9f0b858 100644
> --- i/include/linux/pm.h
> +++ w/include/linux/pm.h
> @@ -592,6 +592,7 @@ struct dev_pm_info {
>  	bool			is_suspended:1;	/* Ditto */
>  	bool			is_noirq_suspended:1;
>  	bool			is_late_suspended:1;
> +	bool			no_pm_required:1;
>  	bool			early_init:1;	/* Owned by the PM core */
>  	bool			direct_complete:1;	/* Owned by the PM core */
>  	u32			driver_flags;