Re: [PATCH v3 04/13] coresight: etm4x: Update ETM4 driver to use Trace ID API

From: Mike Leach
Date: Thu Oct 06 2022 - 09:47:40 EST


Hi Suzuki,

On Mon, 3 Oct 2022 at 10:37, Suzuki K Poulose <suzuki.poulose@xxxxxxx> wrote:
>
> On 03/10/2022 10:31, Suzuki K Poulose wrote:
> > On 09/08/2022 23:33, Mike Leach wrote:
> >> The trace ID API is now used to allocate trace IDs for ETM4.x / ETE
> >> devices.
> >>
> >> For perf sessions, these will be allocated on enable, and released on
> >> disable.
> >>
> >> For sysfs sessions, these will be allocated on enable, but only released
> >> on reset. This allows the sysfs session to interrogate the Trace ID used
> >> after the session is over - maintaining functional consistency with the
> >> previous allocation scheme.
> >>
> >> The trace ID will also be allocated on read of the mgmt/trctraceid file.
> >> This ensures that if perf or sysfs read this before enabling trace, the
> >> value will be the one used for the trace session.
> >>
> >> Trace ID initialisation is removed from the _probe() function.
> >>
> >> Signed-off-by: Mike Leach <mike.leach@xxxxxxxxxx>
> >> ---
> >> .../coresight/coresight-etm4x-core.c | 79 +++++++++++++++++--
> >> .../coresight/coresight-etm4x-sysfs.c | 27 ++++++-
> >> drivers/hwtracing/coresight/coresight-etm4x.h | 3 +
> >> 3 files changed, 100 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> index cf249ecad5a5..b4fb28ce89fd 100644
> >> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> @@ -42,6 +42,7 @@
> >> #include "coresight-etm4x-cfg.h"
> >> #include "coresight-self-hosted-trace.h"
> >> #include "coresight-syscfg.h"
> >> +#include "coresight-trace-id.h"
> >> static int boot_enable;
> >> module_param(boot_enable, int, 0444);
> >> @@ -234,6 +235,50 @@ static int etm4_trace_id(struct coresight_device
> >> *csdev)
> >> return drvdata->trcid;
> >> }
> >> +int etm4_read_alloc_trace_id(struct etmv4_drvdata *drvdata)
> >> +{
> >> + int trace_id;
> >> +
> >> + /*
> >> + * This will allocate a trace ID to the cpu,
> >> + * or return the one currently allocated.
> >> + */
> >> + /* trace id function has its own lock */
> >> + trace_id = coresight_trace_id_get_cpu_id(drvdata->cpu);
> >> + if (IS_VALID_ID(trace_id))
> >> + drvdata->trcid = (u8)trace_id;
> >> + else
> >> + dev_err(&drvdata->csdev->dev,
> >> + "Failed to allocate trace ID for %s on CPU%d\n",
> >> + dev_name(&drvdata->csdev->dev), drvdata->cpu);
> >> + return trace_id;
> >> +}
> >> +
> >> +static int etm4_set_current_trace_id(struct etmv4_drvdata *drvdata)
> >> +{
> >> + int trace_id;
> >> +
> >> + /*
> >> + * Set the currently allocated trace ID - perf allocates IDs
> >> + * as part of setup_aux for all CPUs it may use.
> >> + */
> >> + trace_id = coresight_trace_id_read_cpu_id(drvdata->cpu);
> >> + if (IS_VALID_ID(trace_id)) {
> >> + drvdata->trcid = (u8)trace_id;
> >> + return 0;
> >> + }
> >> +
> >> + dev_err(&drvdata->csdev->dev, "Failed to set trace ID for %s on
> >> CPU%d\n",
> >> + dev_name(&drvdata->csdev->dev), drvdata->cpu);
> >> +
> >> + return -EINVAL;
> >> +}
> >
> >
> >> +
> >> +void etm4_release_trace_id(struct etmv4_drvdata *drvdata)
> >> +{
> >> + coresight_trace_id_put_cpu_id(drvdata->cpu);
> >> +}
> >> +
> >> struct etm4_enable_arg {
> >> struct etmv4_drvdata *drvdata;
> >> int rc;
> >> @@ -729,6 +774,15 @@ static int etm4_enable_perf(struct
> >> coresight_device *csdev,
> >> ret = etm4_parse_event_config(csdev, event);
> >> if (ret)
> >> goto out;
> >> +
> >> + /*
> >> + * perf allocates cpu ids as part of setup - device needs to use
> >> + * the allocated ID.
> >> + */
> >> + ret = etm4_set_current_trace_id(drvdata);
> >
> > So, when do we allocate an id in perf mode ? As far as I can see, this
> > should be the same as etm4_read_alloc_trace_id() ? Why are they any
> > different ?
> >

Trace IDs are allocated in etm_setup_aux(), then released in free_event_data().
This way the value are guaranteed to remain constant for the entire
session,. We cannot release an ID when a given ETM stops tracing, as
it may restart later for the same perf session.
In between these two events perf takes its own locks - for whatever
reason. If we use read_alloc trace id - which takes that allocation /
release lock in the ID system then we see at lot of
LOCKDEP issues at this point.

Hence we do a safe unlocked read of the value - perf will never
release the value till after the session is completed and the event is
released.

> >> + if (ret < 0)
> >> + goto out;
> >> +
> >> /* And enable it */
> >> ret = etm4_enable_hw(drvdata);
> >> @@ -753,6 +807,11 @@ static int etm4_enable_sysfs(struct
> >> coresight_device *csdev)
> >> spin_lock(&drvdata->spinlock);
> >> + /* sysfs needs to read and allocate a trace ID */
> >> + ret = etm4_read_alloc_trace_id(drvdata);
> >> + if (ret < 0)
> >> + goto unlock_sysfs_enable;
> >> +
> >> /*
> >> * Executing etm4_enable_hw on the cpu whose ETM is being enabled
> >> * ensures that register writes occur when cpu is powered.
> >> @@ -764,6 +823,11 @@ static int etm4_enable_sysfs(struct
> >> coresight_device *csdev)
> >> ret = arg.rc;
> >> if (!ret)
> >> drvdata->sticky_enable = true;
> >> +
> >> + if (ret)
> >> + etm4_release_trace_id(drvdata);
> >> +
> >> +unlock_sysfs_enable:
> >> spin_unlock(&drvdata->spinlock);
> >> if (!ret)
> >> @@ -895,6 +959,8 @@ static int etm4_disable_perf(struct
> >> coresight_device *csdev,
> >> /* TRCVICTLR::SSSTATUS, bit[9] */
> >> filters->ssstatus = (control & BIT(9));
> >> + /* The perf event will release trace ids when it is destroyed */
> >> +
> >
> > At this patch level, there is no release of trace id ? Is that missed in
> > this patch ? Or am I missing something ?
>

See above - perf allocation is handled in coresight-etm-perf.c.


> I think the above change only comes in PATCH 7. May be that patch needs
> to be rearranged in order ? Otherwise git-bisect can break running a
> perf session on cs_etm, with missing traceid.
>

This is probably true - we can move the perf allocation to before the
ETM changes as the extended backward compatibility introduced this set
should permit it.


> Suzuki

Thanks for the review

Mike

--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK