Re: [PATCH V2] coresight: tmc: Make etr buffer mode user configurable from sysfs
From: Suzuki K Poulose
Date: Wed Aug 30 2023 - 14:52:47 EST
Cc: Sudeep
Hi Steve
On 28/08/2023 17:35, Steve Clevenger wrote:
Hi Suzuki,
On 8/27/2023 2:35 PM, Suzuki K Poulose wrote:
Hi Steve
On 26/08/2023 01:14, Steve Clevenger wrote:
Unfortunately, I tested with the original patch not [PATCH V2]. I've
remedied this. My results below:
[root@sut01sys-b212 linux]# cat
/sys/devices/system/cpu/cpu123/ARMHC501\:23/tmc_etr35/buf_modes_available
auto flat catu
[root@sut01sys-b212 linux]# cat
/sys/devices/system/cpu/cpu123/ARMHC501\:23/tmc_etr35/buf_mode_preferred
auto
[root@sut01sys-b212 linux]# echo "catu" >
/sys/devices/system/cpu/cpu123/ARMHC501\:23/tmc_etr35/buf_mode_preferred
[root@sut01sys-b212 linux]# cat
/sys/devices/system/cpu/cpu123/ARMHC501\:23/tmc_etr35/buf_mode_preferred
catu
As with the V1 patch, auto defaults to catu.
I expected to see tmc-sg (former default) as an available mode, but do
not. As I recall, the buffer mode defaulted to ETR scatter-gather prior
to this patch. Must this capability now be explicitly advertised? I've
seen this done as "arm,scatter-gather" in device trees, but not used by
Ampere. Perhaps someone can enlighten me.
Yes, you must add that property to the TMC-ETR node (for both DT and
ACPI). In the past, almost all of the TMC-ETRs (except Juno board)
locked up the system while using the SG mode (due to the interconnect
issues, something to do with the transaction). Thus, we decided to
add a property explicitly enabling this for a given platform.
When you mentioned, it was using TMC-ETR SG mode, how did you verify
this ? Please be aware that the table allocation code etc are shared
by both TMC-SG and CATU.
You might recall how this started. I had no way to test the CATU due to
the order the ETR modes defaulted (Flat, ETR-SG, CATU). For test
purposes, I programmatically swapped the ETR-SG/CATU order and could
then verify CATU operation by the driver calling into CATU code. This
So, were you using the DT based boot for the above runs ?
suggests Flat mode was bypassed, and the driver defaulted to ETR-SG
prior to this hack. This didn't offer the user any control, hence my
feature request. Note that most of the early Ampere self-hosted trace
collection used ETR-SG. Now I can't select it.
How is this property described in the ACPI? The "ACPI for CoreSight™ 1.1
Platform Design Document" (DEN0067) doesn't describe this.
This is not specified in the ACPI platform design document. I can get
it fixed. Ideally we need a property describing that the scatter-gather
mode is safe to use.
DT uses "arm,scatter-gather" property [0] and this is what we now expect
in the ACPI based systems too.
https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/arm/arm,coresight-tmc.yaml#:~:text=71-,72,-73
Does it sound fine ?
Suzuki
Thanks,
Steve
Kind regards
Suzuki
Steve C.
On 8/23/2023 4:10 PM, Steve Clevenger wrote:
Here's some quick feedback. My system shows two modes available; auto
catu
etr_buf_mode_current is writable. I expected to see tmc-sg (former
default) listed in etr_buf_modes_available but it doesn't show up.
Note that both the auto and catu etr_buf_mode_current settings default
to catu. My understanding is auto should revert to the default behavior.
On my system the default was tmc-sg.
More later.
[root@sut01sys-b212 kernel]# cat
/sys/devices/system/cpu/cpu20/ARMHC501\:60/tmc_etr96/etr_buf_modes_available
auto catu
[root@sut01sys-b212 kernel]# cat
/sys/devices/system/cpu/cpu20/ARMHC501\:60/tmc_etr96/etr_buf_mode_current
catu
[root@sut01sys-b212 kernel]# echo "catu" >
/sys/devices/system/cpu/cpu20/ARMHC501\:60/tmc_etr96/etr_buf_mode_current
[root@sut01sys-b212 kernel]# cat
/sys/devices/system/cpu/cpu20/ARMHC501\:60/tmc_etr96/etr_buf_mode_current
catu
Steve C.
On 8/21/2023 12:40 PM, Steve Clevenger wrote:
Hi Suzuki,
I may be able to test it this week. You've already pointed me at the
patch thread(s). The main holdup is I need to merge the 6.6 pending
platform work in order to use the Ampere ACPI. I couldn't get these
patches to apply directly to 6.4 last I tried.
Steve C.
On 8/18/2023 2:39 AM, Suzuki K Poulose wrote:
Cc: Steve
Steve,
Are you able to test this with CATU ?
On 18/08/2023 09:21, Anshuman Khandual wrote:
Currently TMC-ETR automatically selects the buffer mode from all
available
methods in the following sequentially fallback manner - also in that
order.
1. FLAT mode with or without IOMMU
2. TMC-ETR-SG (scatter gather) mode when available
3. CATU mode when available
But this order might not be ideal for all situations. For example if
there
is a CATU connected to ETR, it may be better to use TMC-ETR scatter
gather
method, rather than CATU. But hard coding such order changes will
prevent
us from testing or using a particular mode. This change provides
following
new sysfs tunables for the user to control TMC-ETR buffer mode
explicitly,
if required. This adds following new sysfs files for buffer mode
selection
purpose explicitly in the user space.
/sys/bus/coresight/devices/tmc_etr<N>/buf_modes_available
/sys/bus/coresight/devices/tmc_etr<N>/buf_mode_preferred
$ cat buf_modes_available
auto flat tmc-sg catu ------------------> Supported TMC-ETR buffer
modes
$ echo catu > buf_mode_preferred -------> Explicit buffer mode
request
But explicit user request has to be within supported ETR buffer modes
only.
These sysfs interface files are exclussive to ETR, and hence these
are
not
available for other TMC devices such as ETB or ETF etc.
A new auto' mode (i.e ETR_MODE_AUTO) has been added to help fallback
to the
existing default behaviour, when user provided preferred buffer mode
fails.
ETR_MODE_FLAT and ETR_MODE_AUTO are always available as preferred
modes.
Cc: Suzuki K Poulose <suzuki.poulose@xxxxxxx>
Cc: Mike Leach <mike.leach@xxxxxxxxxx>
Cc: James Clark <james.clark@xxxxxxx>
Cc: Leo Yan <leo.yan@xxxxxxxxxx>
Cc: Alexander Shishkin <alexander.shishkin@xxxxxxxxxxxxxxx>
Cc: coresight@xxxxxxxxxxxxxxxx
Cc: linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
Cc: linux-kernel@xxxxxxxxxxxxxxx
Signed-off-by: Anshuman Khandual <anshuman.khandual@xxxxxxx>
---
This applies on v6.5-rc6
Changes in V2:
- Renamed sysfs file etr_buf_modes_available as buf_modes_available
- Renamed sysfs file buf_mode_current as buf_mode_preferred
- Renamed etr_supports_flat_mode() as etr_can_use_flat_mode()
- Renamed coresight_tmc_groups[] as coresight_etf_groups[]
- Reused coresight_tmc_group[] for trigger_cntr and buffer_size
- Fallback trying ETR_MODE_AUTO when user preferred mode fails
- Moved ETR sysfs details into coresight-tmc-etr.c
- Dropped etr_can_use_flat_mode() check while offering ETR_MODE_FLAT
in sysfs
- Moved struct etr_buf_hw inside coresight-tmc-etr.c
- Moved get_etr_buf_hw() and etr_can_use_flat_mode() inside
coresight-tmc-etr.c
- Updated month in
Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc
Changes in V1:
https://lore.kernel.org/all/20230728084837.276551-1-anshuman.khandual@xxxxxxx/
.../testing/sysfs-bus-coresight-devices-tmc | 16 +++
.../hwtracing/coresight/coresight-tmc-core.c | 15 ++-
.../hwtracing/coresight/coresight-tmc-etr.c | 111
++++++++++++++++--
drivers/hwtracing/coresight/coresight-tmc.h | 3 +
4 files changed, 131 insertions(+), 14 deletions(-)
Looks good to me.
Suzuki