Re: [PATCH v7 28/28] coresight: Add support for v8.4 SelfHosted tracing

[Mike Leach]

Hi Suzuki,

On Fri, 12 Feb 2021 at 15:36, Suzuki K Poulose <suzuki.poulose@xxxxxxx> wrote:
>
> Hi Mike
>
> On 2/12/21 10:34 AM, Mike Leach wrote:
> > Hi Mathieu, Suzuki,
> >
> > Sorry for the really late response on this patch, but I noticed a
> > problem while doing a review of the ETE / TRBE set. (TRBE specs
> > mention TRFCR_ELx, so I was confirming a couple of things).
> >
> > On Sun, 10 Jan 2021 at 22:49, Suzuki K Poulose <suzuki.poulose@xxxxxxx> wrote:
> >>
> >> From: Jonathan Zhou <jonathan.zhouwen@xxxxxxxxxx>
> >>
> >> v8.4 tracing extensions added support for trace filtering controlled
> >> by TRFCR_ELx. This must be programmed to allow tracing at EL1/EL2 and
> >> EL0. The timestamp used is the virtual time. Also enable CONTEXIDR_EL2
> >> tracing if we are running the kernel at EL2.
> >>
> >> Cc: Catalin Marinas <catalin.marinas@xxxxxxx>
> >> Cc: Mike Leach <mike.leach@xxxxxxxxxx>
> >> Cc: Will Deacon <will@xxxxxxxxxx>
> >> Reviewed-by: Mathieu Poirier <mathieu.poirier@xxxxxxxxxx>
> >> Signed-off-by: Jonathan Zhou <jonathan.zhouwen@xxxxxxxxxx>
> >> [ Move the trace filtering setup etm_init_arch_data() and
> >>   clean ups]
> >> Signed-off-by: Suzuki K Poulose <suzuki.poulose@xxxxxxx>
> >> ---
> >>   .../coresight/coresight-etm4x-core.c          | 25 +++++++++++++++++++
> >>   1 file changed, 25 insertions(+)
> >>
> >> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> index 3d3165dd09d4..18c1a80abab8 100644
> >> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> >> @@ -859,6 +859,30 @@ static bool etm4_init_csdev_access(struct etmv4_drvdata *drvdata,
> >>          return false;
> >>   }
> >>
> >> +static void cpu_enable_tracing(void)
> >> +{
> >> +       u64 dfr0 = read_sysreg(id_aa64dfr0_el1);
> >> +       u64 trfcr;
> >> +
> >> +       if (!cpuid_feature_extract_unsigned_field(dfr0, ID_AA64DFR0_TRACE_FILT_SHIFT))
> >> +               return;
> >> +
> >> +       /*
> >> +        * If the CPU supports v8.4 SelfHosted Tracing, enable
> >> +        * tracing at the kernel EL and EL0, forcing to use the
> >> +        * virtual time as the timestamp.
> >> +        */
> >> +       trfcr = (TRFCR_ELx_TS_VIRTUAL |
> >> +                TRFCR_ELx_ExTRE |
> >> +                TRFCR_ELx_E0TRE);
> >> +
> >> +       /* If we are running at EL2, allow tracing the CONTEXTIDR_EL2. */
> >> +       if (is_kernel_in_hyp_mode())
> >> +               trfcr |= TRFCR_EL2_CX;
> >> +
> >
> > This is wrong - CX bit is present on TRFCR_EL2, not TRFCR_EL1.
>
> Why is this wrong ? We do this only when we are in EL2.
>

Sorry  - must have been looking at an older version of the ARMARM when
I looked for EL1 registers that are aliased to EL2.  So this does
indeed work!

> > Moreover, TRFCR_EL2 has a separate enables for tracing at EL0 and EL2.
> >
>
> True, that is for EL0&2 translation regimes. i.e, tracing EL0 with
> the kernel running at EL2. But bits TRFCR_EL2.E2TRE == TRFCR_EL1.E1TRE
> If notice, we name the bit TRFCR_ELx_ExTRE. And E0TRE == E0HTRE.
>
> So we do the following :
>
>    1) When kernel running at EL2:
>      Enable tracing at EL2 and EL0 and context tracking
>    2) When kernel running at EL1:
>      Enable tracing at EL1 and EL0.
>
>
> > Secondly - is this correct in principal?  Should the driver not be
> > reading the access it is permitted by the kernel, rather than giving
> > itself unfettered access to trace where it wants to.
>
> I dont follow the "access permitted by the kernel" here. What are we referrring to ?
>

By that I mean that as I suggest below this should be controlled by
what we could call the hypervisor, rather than a driver.

> > Surely TRFCR_ELx  levels should be chosen in KConfig  and then should
> > be set up in kernel initialisation?
>
> I disagree with yet another Kconfig. This basic requirement for
> enabling the trace collection. It is not something that we can optionally
> use from the architecture. So we should transparently do the right
> thing for making sure that we set up the system for something that
> didn't require any other steps. Or in other words, if we add a Kconfig
> option for TRFCR programming, if someone forgets to select it
> when they upgraded the kernel they are in for a surprisingly long
> debugging to find why the trace doesnt work.
>
> As for the TRFCR programming, we have two choices. etm4x driver
> or generic boot up for the CPU. I preferred to do this in the
> driver as we can enable it only if trace drivers are available.
>

The point is that TRFCR are not part of the controlling registers for
the ETE or any trace source device. The architecture manual seems to
regard them as being controlled by the hypervisor, rather than the PE
trace device. This implies that the control feature is designed to be
independent from the trace generation features.

I thought they were there to allow virtualisation code to determine
what gets traced and what is prohibited, and what view the trace sees
of the clock. If you simple switch everything on from the driver and
control the ELs traced from the ETE / ETM registers then what are they
there for?

This solution could be a first pass at this to get trace working, but
I think it will have to change in future.

Regards

Mike



> Cheers
> Suzuki
>
> >
> > Regards
> >
> > Mike
> >
> >
> >
> >> +       write_sysreg_s(trfcr, SYS_TRFCR_EL1);
> >> +}
> >> +
> >>   static void etm4_init_arch_data(void *info)
> >>   {
> >>          u32 etmidr0;
> >> @@ -1044,6 +1068,7 @@ static void etm4_init_arch_data(void *info)
> >>          /* NUMCNTR, bits[30:28] number of counters available for tracing */
> >>          drvdata->nr_cntr = BMVAL(etmidr5, 28, 30);
> >>          etm4_cs_lock(drvdata, csa);
> >> +       cpu_enable_tracing();
> >>   }
> >>
> >>   static inline u32 etm4_get_victlr_access_type(struct etmv4_config *config)
> >> --
> >> 2.24.1
> >>
> >
> >
> > --
> > Mike Leach
> > Principal Engineer, ARM Ltd.
> > Manchester Design Centre. UK
> >
>


--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK