RE: [RFC 1/2] remoteproc: sysfs: authorize rproc shutdown when rproc is crashed

[Loic PALLARDY]

Hi Bjorn,

> -----Original Message-----
> From: Bjorn Andersson <bjorn.andersson@xxxxxxxxxx>
> Sent: jeudi 12 mars 2020 00:27
> To: Loic PALLARDY <loic.pallardy@xxxxxx>
> Cc: ohad@xxxxxxxxxx; mathieu.poirier@xxxxxxxxxx; linux-
> remoteproc@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; Arnaud
> POULIQUEN <arnaud.pouliquen@xxxxxx>; benjamin.gaignard@xxxxxxxxxx;
> Fabien DESSENNE <fabien.dessenne@xxxxxx>; s-anna@xxxxxx
> Subject: Re: [RFC 1/2] remoteproc: sysfs: authorize rproc shutdown when
> rproc is crashed
> 
> On Wed 11 Mar 03:54 PDT 2020, Loic Pallardy wrote:
> 
> > When remoteproc recovery is disabled and rproc crashed, user space
> > client has no way to reboot co-processor except by a complete platform
> > reboot.
> > Indeed rproc_shutdown() is called by sysfs state_store() only is rproc
> > state is RPROC_RUNNING.
> >
> > This patch offers the possibility to shutdown the co-processor if
> > it is in RPROC_CRASHED state and so to restart properly co-processor
> > from sysfs interface.
> >
> 
> I did recently run into a similar problem when I fed my remoteproc
> faulty firmware, which lead to it recovering immediately upon boot. The
> amount of time spent in !CRASHED state was minimal, so I didn't have any
> way to stop the remoteproc.
> 
> > Signed-off-by: Loic Pallardy <loic.pallardy@xxxxxx>
> > ---
> >  drivers/remoteproc/remoteproc_core.c  | 2 +-
> >  drivers/remoteproc/remoteproc_sysfs.c | 2 +-
> >  2 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/remoteproc/remoteproc_core.c
> b/drivers/remoteproc/remoteproc_core.c
> > index 097f33e4f1f3..7ac87a75cd1b 100644
> > --- a/drivers/remoteproc/remoteproc_core.c
> > +++ b/drivers/remoteproc/remoteproc_core.c
> > @@ -1812,7 +1812,7 @@ void rproc_shutdown(struct rproc *rproc)
> >  	if (!atomic_dec_and_test(&rproc->power))
> >  		goto out;
> >
> > -	ret = rproc_stop(rproc, false);
> > +	ret = rproc_stop(rproc, rproc->state == RPROC_CRASHED);
> 
> Afaict this is unrelated to the problem you're describing in the commit
> message.
Right, it is because now rproc_shudown could be could in a context where rproc is in RPROC_CRASHED state and so false is no more the default value.
Could be split in another patch.

> 
> >  	if (ret) {
> >  		atomic_inc(&rproc->power);
> >  		goto out;
> > diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> b/drivers/remoteproc/remoteproc_sysfs.c
> > index 7f8536b73295..1029458a4678 100644
> > --- a/drivers/remoteproc/remoteproc_sysfs.c
> > +++ b/drivers/remoteproc/remoteproc_sysfs.c
> > @@ -101,7 +101,7 @@ static ssize_t state_store(struct device *dev,
> >  		if (ret)
> >  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
> >  	} else if (sysfs_streq(buf, "stop")) {
> > -		if (rproc->state != RPROC_RUNNING)
> > +		if (rproc->state != RPROC_RUNNING && rproc->state !=
> RPROC_CRASHED)
> 
> Analogous to the problem reported by Alex here
> https://patchwork.kernel.org/patch/11413161/ the handling of stop seems
> racy.
> 
> In particular, I believe you're failing to protect against a race
> with a just scheduled rproc_crash_handler_work() being executed after
> the mutex_unlock() in rproc_shutdown()...
> 
> With Alex fix that should be less of a problem though...
Thanks for pointing me Alex's patch. But I don't think it is exactly the same issue as it concerns the recovery procedure itself.
In my case, the recovery is disabled. On a crash detection, rproc->state is simply set to RPROC_CRASHED
and recovery is not  triggered.
Without client action, rproc will stay forever in RPROC_CRASHED test.
Today without this modification, it is not possible to shutdown rproc properly, putting coprocessor under reset,  disabling clocks...

Regards,
Loic

> 
> Regards,
> Bjorn
> 
> >  			return -EINVAL;
> >
> >  		rproc_shutdown(rproc);
> > --
> > 2.7.4
> >