Re: [PATCH 3/3] remoteproc/k3-dsp: Add support for L2RAM loading on C66x DSPs

[Suman Anna]

Hi Mathieu,

On 4/28/20 3:09 PM, Mathieu Poirier wrote:
> On Tue, 28 Apr 2020 at 13:58, Mathieu Poirier
> <mathieu.poirier@xxxxxxxxxx> wrote:
> > On Wed, Mar 25, 2020 at 03:18:39PM -0500, Suman Anna wrote:
> > > The resets for the DSP processors on K3 SoCs are managed through the
> > > Power and Sleep Controller (PSC) module. Each DSP typically has two
> > > resets - a global module reset for powering on the device, and a local
> > > reset that affects only the CPU while allowing access to the other
> > > sub-modules within the DSP processor sub-systems.
> > >
> > > The C66x DSPs have two levels of internal RAMs that can be used to
> > > boot from, and the firmware loading into these RAMs require the
> > > local reset to be asserted with the device powered on/enabled using
> > > the module reset. Enhance the K3 DSP remoteproc driver to add support
> > > for loading into the internal RAMs. The local reset is deasserted on
> > > SoC power-on-reset, so logic has to be added in probe in remoteproc
> > > mode to balance the remoteproc state-machine.
> > >
> > > Note that the local resets are a no-op on C71x cores, and the hardware
> > > does not supporting loading into its internal RAMs.
> > >
> > > Signed-off-by: Suman Anna <s-anna@xxxxxx>
> > > ---
> > >   drivers/remoteproc/ti_k3_dsp_remoteproc.c | 82 +++++++++++++++++++++++
> > >   1 file changed, 82 insertions(+)
> > >
> > > diff --git a/drivers/remoteproc/ti_k3_dsp_remoteproc.c b/drivers/remoteproc/ti_k3_dsp_remoteproc.c
> > > index fd0d84f46f90..7b712ef74611 100644
> > > --- a/drivers/remoteproc/ti_k3_dsp_remoteproc.c
> > > +++ b/drivers/remoteproc/ti_k3_dsp_remoteproc.c
> > > @@ -175,6 +175,9 @@ static int k3_dsp_rproc_reset(struct k3_dsp_rproc *kproc)
> > >                return ret;
> > >        }
> > >
> > > +     if (kproc->data->uses_lreset)
> > > +             return ret;
> > > +
> > >        ret = kproc->ti_sci->ops.dev_ops.put_device(kproc->ti_sci,
> > >                                                    kproc->ti_sci_id);
> > >        if (ret) {
> > > @@ -192,6 +195,9 @@ static int k3_dsp_rproc_release(struct k3_dsp_rproc *kproc)
> > >        struct device *dev = kproc->dev;
> > >        int ret;
> > >
> > > +     if (kproc->data->uses_lreset)
> > > +             goto lreset;
> > > +
> > >        ret = kproc->ti_sci->ops.dev_ops.get_device(kproc->ti_sci,
> > >                                                   kproc->ti_sci_id);
> > >        if (ret) {
> > > @@ -199,6 +205,7 @@ static int k3_dsp_rproc_release(struct k3_dsp_rproc *kproc)
> > >                return ret;
> > >        }
> > >
> > > +lreset:
> > >        ret = reset_control_deassert(kproc->reset);
> > >        if (ret) {
> > >                dev_err(dev, "local-reset deassert failed, ret = %d\n", ret);
> > > @@ -210,6 +217,63 @@ static int k3_dsp_rproc_release(struct k3_dsp_rproc *kproc)
> > >        return ret;
> > >   }
> > >
> > > +/*
> > > + * The C66x DSP cores have a local reset that affects only the CPU, and a
> > > + * generic module reset that powers on the device and allows the DSP internal
> > > + * memories to be accessed while the local reset is asserted. This function is
> > > + * used to release the global reset on C66x DSPs to allow loading into the DSP
> > > + * internal RAMs. The .prepare() ops is invoked by remoteproc core before any
> > > + * firmware loading, and is followed by the .start() ops after loading to
> > > + * actually let the C66x DSP cores run. The local reset on C71x cores is a
> > > + * no-op and the global reset cannot be released on C71x cores until after
> > > + * the firmware images are loaded, so this function does nothing for C71x cores.
> > > + */
> > > +static int k3_dsp_rproc_prepare(struct rproc *rproc)
> > > +{
> > > +     struct k3_dsp_rproc *kproc = rproc->priv;
> > > +     struct device *dev = kproc->dev;
> > > +     int ret;
> > > +
> > > +     /* local reset is no-op on C71x processors */
> > > +     if (!kproc->data->uses_lreset)
> > > +             return 0;
> >
> > In k3_dsp_rproc_release() the condition is "if (kproc->data->uses_lreset)" and
> > here it is the opposite, which did a good job at getting me confused.

Do you prefer I add a comment there? It needs to bail out there since 
the get_device portion would be executed here.

> > Taking a step back, I assume c71 DSPs will have their own k3_dsp_dev_data where
> > the users_lreset flag will be false.  

Yes.

In that case I think it would make the
> > code easier to understand if the k3_dsp_rproc_ops was declared without the
> > .prepare and .unprepare.  In probe(), if data->uses_lreset is true then
> > k3_dsp_rproc_prepare() and k3_dsp_rproc_unprepare() are set.

Yeah, ok, that will avoid the confusion and limit the 
prepare()/unprepare() only for C66 DSPs.

> >
>
> I forgot... Since this is a C71 related change, was there a reason to
> lump it with the C66 set?  If not I would simply move that to the C71
> work.

OK, I can remove this logic here, and add the prepare()/unprepare() 
conditionally for C66x in the C71 patch.

> > I am done reviewing this set.

Thanks for all the review comments.

regards
Suman

> > Thanks,
> > Mathieu
> >
> > > +
> > > +     ret = kproc->ti_sci->ops.dev_ops.get_device(kproc->ti_sci,
> > > +                                                 kproc->ti_sci_id);
> > > +     if (ret)
> > > +             dev_err(dev, "module-reset deassert failed, cannot enable internal RAM loading, ret = %d\n",
> > > +                     ret);
> > > +
> > > +     return ret;
> > > +}
> > > +
> > > +/*
> > > + * This function implements the .unprepare() ops and performs the complimentary
> > > + * operations to that of the .prepare() ops. The function is used to assert the
> > > + * global reset on applicable C66x cores. This completes the second portion of
> > > + * powering down the C66x DSP cores. The cores themselves are only halted in the
> > > + * .stop() callback through the local reset, and the .unprepare() ops is invoked
> > > + * by the remoteproc core after the remoteproc is stopped to balance the global
> > > + * reset.
> > > + */
> > > +static int k3_dsp_rproc_unprepare(struct rproc *rproc)
> > > +{
> > > +     struct k3_dsp_rproc *kproc = rproc->priv;
> > > +     struct device *dev = kproc->dev;
> > > +     int ret;
> > > +
> > > +     /* local reset is no-op on C71x processors */
> > > +     if (!kproc->data->uses_lreset)
> > > +             return 0;
> > > +
> > > +     ret = kproc->ti_sci->ops.dev_ops.put_device(kproc->ti_sci,
> > > +                                                 kproc->ti_sci_id);
> > > +     if (ret)
> > > +             dev_err(dev, "module-reset assert failed, ret = %d\n", ret);
> > > +
> > > +     return ret;
> > > +}
> > > +
> > >   /*
> > >    * Power up the DSP remote processor.
> > >    *
> > > @@ -353,6 +417,8 @@ static void *k3_dsp_rproc_da_to_va(struct rproc *rproc, u64 da, size_t len)
> > >   }
> > >
> > >   static const struct rproc_ops k3_dsp_rproc_ops = {
> > > +     .prepare        = k3_dsp_rproc_prepare,
> > > +     .unprepare      = k3_dsp_rproc_unprepare,
> > >        .start          = k3_dsp_rproc_start,
> > >        .stop           = k3_dsp_rproc_stop,
> > >        .kick           = k3_dsp_rproc_kick,
> > > @@ -644,6 +710,22 @@ static int k3_dsp_rproc_probe(struct platform_device *pdev)
> > >                goto disable_clk;
> > >        }
> > >
> > > +     /*
> > > +      * ensure the DSP local reset is asserted to ensure the DSP doesn't
> > > +      * execute bogus code in .prepare() when the module reset is released.
> > > +      */
> > > +     if (data->uses_lreset) {
> > > +             ret = reset_control_status(kproc->reset);
> > > +             if (ret < 0) {
> > > +                     dev_err(dev, "failed to get reset status, status = %d\n",
> > > +                             ret);
> > > +                     goto release_mem;
> > > +             } else if (ret == 0) {
> > > +                     dev_warn(dev, "local reset is deasserted for device\n");
> > > +                     k3_dsp_rproc_reset(kproc);
> > > +             }
> > > +     }
> > > +
> > >        ret = rproc_add(rproc);
> > >        if (ret) {
> > >                dev_err(dev, "failed to add register device with remoteproc core, status = %d\n",
> > > --
> > > 2.23.0