RE: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features

From: Moger, Babu
Date: Fri Oct 07 2022 - 11:52:08 EST

Next message: Krzysztof Kozlowski: "Re: [PATCH v6 06/17] dt-bindings: mfd: amd,pensando-elbasr: Add AMD Pensando Elba System Resource chip"
Previous message: Guenter Roeck: "Re: [PATCH v2] watchdog: w83627hf_wdt: add bootstatus support"
In reply to: Bagas Sanjaya: "Re: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

[AMD Official Use Only - General]

Hi Sanjaya,

> -----Original Message-----
> From: Bagas Sanjaya <bagasdotme@xxxxxxxxx>
> Sent: Friday, October 7, 2022 3:33 AM
> To: Moger, Babu <Babu.Moger@xxxxxxx>; corbet@xxxxxxx;
> reinette.chatre@xxxxxxxxx; tglx@xxxxxxxxxxxxx; mingo@xxxxxxxxxx;
> bp@xxxxxxxxx
> Cc: fenghua.yu@xxxxxxxxx; dave.hansen@xxxxxxxxxxxxxxx; x86@xxxxxxxxxx;
> hpa@xxxxxxxxx; paulmck@xxxxxxxxxx; akpm@xxxxxxxxxxxxxxxxxxxx;
> quic_neeraju@xxxxxxxxxxx; rdunlap@xxxxxxxxxxxxx;
> damien.lemoal@xxxxxxxxxxxxxxxxxx; songmuchun@xxxxxxxxxxxxx;
> peterz@xxxxxxxxxxxxx; jpoimboe@xxxxxxxxxx; pbonzini@xxxxxxxxxx;
> chang.seok.bae@xxxxxxxxx; pawan.kumar.gupta@xxxxxxxxxxxxxxx;
> jmattson@xxxxxxxxxx; daniel.sneddon@xxxxxxxxxxxxxxx; Das1, Sandipan
> <Sandipan.Das@xxxxxxx>; tony.luck@xxxxxxxxx; james.morse@xxxxxxx;
> linux-doc@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx;
> eranian@xxxxxxxxxx
> Subject: Re: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features
>
> On 9/28/22 03:25, Babu Moger wrote:
> > New AMD processors can now support following QoS features.
> >
> > 1. Slow Memory Bandwidth Allocation (SMBA)
> > With this feature, the QOS enforcement policies can be applied
> > to the external slow memory connected to the host. QOS enforcement
> > is accomplished by assigning a Class Of Service (COS) to a processor
> > and specifying allocations or limits for that COS for each resource
> > to be allocated.
> >
> > Currently, CXL.memory is the only supported "slow" memory device. With
> > the support of SMBA feature the hardware enables bandwidth allocation
> > on the slow memory devices.
> >
> > 2. Bandwidth Monitoring Event Configuration (BMEC)
> > The bandwidth monitoring events mbm_total_event and mbm_local_event
> > are set to count all the total and local reads/writes respectively.
> > With the introduction of slow memory, the two counters are not enough
> > to count all the different types are memory events. With the feature
> > BMEC, the users have the option to configure mbm_total_event and
> > mbm_local_event to count the specific type of events.
> >
> > Following are the bitmaps of events supported.
> > Bits Description
> > 6 Dirty Victims from the QOS domain to all types of memory
> > 5 Reads to slow memory in the non-local NUMA domain
> > 4 Reads to slow memory in the local NUMA domain
> > 3 Non-temporal writes to non-local NUMA domain
> > 2 Non-temporal writes to local NUMA domain
> > 1 Reads to memory in the non-local NUMA domain
> > 0 Reads to memory in the local NUMA domain
> >
> > This series adds support for these features.
> >
> > Feature description is available in the specification, "AMD64
> > Technology Platform Quality of Service Extensions, Revision: 1.03 Publication
> # 56375 Revision: 1.03 Issue Date: February 2022".
> >
> > Link:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> > amd.com%2Fen%2Fsupport%2Ftech-docs%2Famd64-technology-platform-
> quality
> > -service-
> extensions&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc806
> >
> 9ca2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%
> 7C0%7C
> >
> 638007284215759374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> AiLCJQIjo
> >
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdat
> a=qAZS
> > ze2Mbg24Z0%2BX0GN4yrVO2ooQqQyum7NUjwIGg5o%3D&reserved=0
> > Link:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugz
> >
> illa.kernel.org%2Fshow_bug.cgi%3Fid%3D206537&data=05%7C01%7Cbab
> u.m
> >
> oger%40amd.com%7Cda5fc8069ca2484b2aef08daa83ea08a%7C3dd8961fe488
> 4e608e
> >
> 11a82d994e183d%7C0%7C0%7C638007284215759374%7CUnknown%7CTWFpb
> GZsb3d8ey
> >
> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> C300
> >
> 0%7C%7C%7C&sdata=8%2BFbGTVfkp%2BCbyyk%2BYa9u0JiHi2YZEVaHiUs
> CBw335g
> > %3D&reserved=0
> >
> > ---
> > v5:
> > Summary of changes.
> > 1. Split the series into two. The first two patches are bug fixes. So, sent them
> separate.
> > 2. The config files mbm_total_config and mbm_local_config are now under
> > /sys/fs/resctrl/info/L3_MON/. Removed these config files from mon
> groups.
> > 3. Ran "checkpatch --strict --codespell" on all the patches. Looks good with
> few known exceptions.
> > 4. Few minor text changes in resctrl.rst file.
> >
> > v4:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> el.org%2Flkml%2F166257348081.1043018.11227924488792315932.stgit%40bm
> oger-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638007284215759374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=qamR1M2sVSo4vE9NVZzFmvIop7YMKNIYHT74NJrbVVk%3D&am
> p;reserved=0
> > Got numerios of comments from Reinette Chatre. Addressed most of them.
> > Summary of changes.
> > 1. Removed mon_configurable under /sys/fs/resctrl/info/L3_MON/.
> > 2. Updated mon_features texts if the BMEC is supported.
> > 3. Added more explanation about the slow memory support.
> > 4. Replaced smp_call_function_many with on_each_cpu_mask call.
> > 5. Removed arch_has_empty_bitmaps
> > 6. Few other text changes.
> > 7. Removed Reviewed-by if the patch is modified.
> > 8. Rebased the patches to latest tip.
> >
> > v3:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> el.org%2Flkml%2F166117559756.6695.16047463526634290701.stgit%40bmoge
> r-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=ogINZslT9yExkkFww4X14XQEg8W8heYBrJB59C50Hqk%3D&
> reserved=0
> > a. Rebased the patches to latest tip. Resolved some conflicts.
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kerne
> l.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftip%2Ftip.git&data=05%
> 7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca2484b2aef08daa83ea08a%
> 7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638007284215915604
> %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
> BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZOK3iwsaq
> 3%2BGAUIXJwn9Thg3cJBxWDMAfu4UqHlo2J4%3D&reserved=0
> > b. Taken care of feedback from Bagas Sanjaya.
> > c. Added Reviewed by from Mingo.
> > Note: I am still looking for comments from Reinette or Fenghua.
> >
> > v2:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> el.org%2Flkml%2F165938717220.724959.10931629283087443782.stgit%40bmo
> ger-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=14viyG9elsn6BYGpDOwrqYQFNOlpyC6oqoJwJm49oV0%3D&
> reserved=0
> > a. Rebased the patches to latest stable tree (v5.18.15). Resolved some
> conflicts.
> > b. Added the patch to fix CBM issue on AMD. This was originally discussed
> >
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> > .kernel.org%2Flkml%2F20220517001234.3137157-1-
> eranian%40google.com%2F&
> >
> amp;data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca2484b2aef0
> 8daa83e
> >
> a08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C6380072842159
> 15604%7
> >
> CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> 6Ik1
> >
> haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BHEvN2PNNYyH
> ohvLg2sbth
> > BQo4cgDj5Vsw9AqGb1Pr8%3D&reserved=0
> >
> > v1:
> >
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> >
> .kernel.org%2Flkml%2F165757543252.416408.13547339307237713464.stgit%4
> 0
> > bmoger-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> >
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638
> >
> 007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> JQIjoiV2
> >
> luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J
> R03cAA
> > 9pzmq0SgNsgAsajOH6LX%2F4s3px3b%2FJ8409Ys%3D&reserved=0
> >
> > Babu Moger (12):
> > x86/cpufeatures: Add Slow Memory Bandwidth Allocation feature flag
> > x86/resctrl: Add a new resource type RDT_RESOURCE_SMBA
> > x86/cpufeatures: Add Bandwidth Monitoring Event Configuration feature
> flag
> > x86/resctrl: Include new features in command line options
> > x86/resctrl: Detect and configure Slow Memory Bandwidth allocation
> > x86/resctrl: Introduce data structure to support monitor configuration
> > x86/resctrl: Add sysfs interface to read mbm_total_bytes event
> configuration
> > x86/resctrl: Add sysfs interface to read mbm_local_bytes event
> configuration
> > x86/resctrl: Add sysfs interface to write mbm_total_bytes event
> configuration
> > x86/resctrl: Add sysfs interface to write mbm_local_bytes event
> configuration
> > x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask()
> > Documentation/x86: Update resctrl_ui.rst for new features
> >
> >
> > .../admin-guide/kernel-parameters.txt | 2 +-
> > Documentation/x86/resctrl.rst | 130 +++++++-
> > arch/x86/include/asm/cpufeatures.h | 2 +
> > arch/x86/kernel/cpu/cpuid-deps.c | 1 +
> > arch/x86/kernel/cpu/resctrl/core.c | 51 ++-
> > arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +-
> > arch/x86/kernel/cpu/resctrl/internal.h | 33 +-
> > arch/x86/kernel/cpu/resctrl/monitor.c | 9 +-
> > arch/x86/kernel/cpu/resctrl/rdtgroup.c | 298 ++++++++++++++++--
> > arch/x86/kernel/cpu/scattered.c | 2 +
> > 10 files changed, 496 insertions(+), 34 deletions(-)
> >
>
> Hi Babu, sorry for having to do public reply to this v5 cover letter due to
> accidentally delete the preview documentation patch for your upcoming v6.
>
> Thanks for privately sending me the preview patch. Seeing it at a glance,
> LGTM. Please send the full v6 series for us to review.
Sure. Will send the whole series.
Thanks
Babu

Next message: Krzysztof Kozlowski: "Re: [PATCH v6 06/17] dt-bindings: mfd: amd,pensando-elbasr: Add AMD Pensando Elba System Resource chip"
Previous message: Guenter Roeck: "Re: [PATCH v2] watchdog: w83627hf_wdt: add bootstatus support"
In reply to: Bagas Sanjaya: "Re: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]