Hi Babu,
On 12/7/2023 8:12 AM, Moger, Babu wrote:
On 12/6/23 12:49, Reinette Chatre wrote:
On 12/6/2023 7:40 AM, Moger, Babu wrote:
On 12/5/23 17:17, Reinette Chatre wrote:
On 11/30/2023 4:57 PM, Babu Moger wrote:
Understood. I would like to start by considering how (if at all) existingThere are two MBM events(total and local) in each group.The new resctrl files in info/ could always be present. For example,Spec saysb. Mount with ABMC supporthmmm ... so this requires the user to mount resctrl, determine if the
#umount /sys/fs/resctrl/
#mount -o abmc -t resctrl resctrl /sys/fs/resctrl/
feature is supported, unmount resctrl, remount resctrl with feature enabled.
Could you please elaborate what prevents this feature from being enabled
without needing to remount resctrl?
"Enabling ABMC: ABMC is enabled by setting L3_QOS_EXT_CFG.ABMC_En=1 (see
Figure 19-7). When the state of ABMC_En is changed, it must be changed to
the updated value on all logical processors in the QOS Domain.
Upon transitions of the ABMC_En the following actions take place:
All ABMC assignable bandwidth counters are reset to 0.
The L3 default mode bandwidth counters are reset to 0.
The L3_QOS_ABMC_CFG MSR is reset to 0."
So, all the monitoring group counters will be reset.
It is technically possible to enable without remount. But ABMC mode
requires few new files(in each group) which I added when mounted with "-o
abmc". Thought it is a better option.
Otherwise we need to add these files when ABMC is supported(not when
enabled). Need to add another file in /sys/fs/resctrl/info/L3_MON to
enable the feature on the fly.
Both are acceptable options. Any thoughts?
user space may want to know how many counters are available before
enabling the feature.
It is not yet obvious to me that this feature requires new files
in monitor groups.
We should provide an interface to assign each event independently.
User can assign only one event in a group. We should also provide an
option assign both the events in the group. This needs to be done at
resctrl group level.
files may be used, thus my example of using mbm_total_bytes, before adding
more files.
...
Thank you for this information. Would reading L3_QOS_ABMC_DSC register onHere are the clarifications from hardware engineer about this.This highlights that this resctrl feature is currently latched to AMD'sHardware still reports it as unavailable. Also, there are some error cases#cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_local_bytesI believe that "Unavailable" already has an accepted meaning within current
Unavailable
interface and is associated with temporary failure. Even the AMD spec states "This
is generally a temporary condition and subsequent reads may succeed". In the
scenario above there is no chance that this counter would produce a value later.
I do not think it is ideal to overload existing interface with different meanings
associated with a new hardware specific feature ... something like "Disabled" seems
more appropriate.
hardware can report unavailable. We may not be able to differentiate that.
ABMC. I do not think we should require that this resctrl feature is backed
by hardware that can support reads of counters that are disabled. A counter
read really only needs to be sent to hardware if it is enabled.
Are the ABMC registers not per CPU? This is unclear to me at this timeConsidering this we may even consider using these files themselves as aI am not sure about this. This is specific to domain 0. This group can
way to enable the counters if they are disabled. For example, just
"echo 1 > /sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes" can be used
have cpus from multiple domains. I think we should have the interface for
all the domains(not for specific domain).
since changelog of patch #13 states it is per-CPU but yet the code
uses smp_call_function_any().
# While configuring the counter, should we have to write (L3_QOS_ABMC_CFG)
on all the logical processors in a domain?
No. In order to configure a specific counter, you only need to write it
on a single logical processor in a domain. Configuring the actual ABMC
counter is a side-effect of the write to this register. And the actual
ABMC counter configuration is a global state.
"Each logical processor implements a separate copy of these registers"
identifies that if you write a 5 to L3_QOS_ABMC_CFG on C0T0, you will not
read a 5 from the L3_QOS_ABMC_CFG register on C1T0.
C1T0 return the configuration written to L3_QOS_ABMC_CFG on C0T0 ?
Yes. It is programmed on all the domains. Separating the domain configuration will require more changes. I am not planning to address in this series.
Even so, you do confirm that the counter configuration is per domain. If I
understand correctly the implementation in this series assumes the counters
are programmed identically on all domains, but theoretically the system can support
domains with different counter configurations. For example, if a resource group
is limited to CPUs in one domain it would be unnecessary to consume the other
domain's counters.
This also ties into what this feature may morph into when considering the
non-ABMC AMD hardware needing similar interface as well as MPAM. I understand
for MPAM that resources are required for a counter but I do not know their
scope.
Reinette