Re: [PATCH v3 0/7] Support L3 Smart Data Cache Injection Allocation Enforcement (SDCIAE)

[Moger, Babu]

Hi Reinette,

On 4/8/2025 8:41 PM, Reinette Chatre wrote:
> Hi Babu,
>
> On 4/8/25 5:41 PM, Moger, Babu wrote:
> > Hi Reinette,
> >
> > On 4/8/2025 4:44 PM, Reinette Chatre wrote:
> > > Hi Babu,
> > >
> > > On 4/7/25 1:12 PM, Moger, Babu wrote:
> > > > On 3/21/25 17:50, Reinette Chatre wrote:
> > > > > On 1/30/25 1:20 PM, Babu Moger wrote:
> > >
> > >
> > > > > >
> > > > >
> > > > > AMD also supports what is exposed to user space as "shareable_bits". According
> > > > > to APM:
> > > > >      Depending on the implementation, some portions of the L3 Cache may be
> > > > >      shared by other system functions or used for some other purpose not
> > > > >      under the control of the PQOS feature set. The L3 Cache Allocation
> > > > >      Sharing Mask returned by CPUID Fn0000_0010_EBX_x1[L3ShareAllocMask] is a
> > > > >      bitmask that represents portions of the L3 that may be shared by those
> > > > >      functions.
> > > >
> > > > Here is the complete text.
> > > >
> > > > The L3 Cache allocation sharing mask (L3ShareAllocMask) returned in EBX by
> > > > CPUID Fn0000_0010 with ECX=1 is a bit vector which represents portions of
> > > > the cache which may be shared with other system entities or used for some
> > > > other purpose not under the control of the QOS feature set. When software
> > > > sets a bit in one of the L3_MASK_n registers at the same bit positions a
> > > > bit in the L3ShareAllocMask, processors executing with the corresponding
> > > > COS will competitively share that portion of the cache with the other
> > > > function. If this mask is all 0’s, then there is no other entity in the
> > > > system competing with the processors for use of the L3 cache.
> > > >
> > > > The "L3ShareAllocMask" is always reported as 0 on AMD systems.
> > > >
> > > > > Could you please include what (if any) the relationship is between the CBM
> > > > > discoverable via Fn0000_0010_EBX_x1[L3ShareAllocMask] and the CBM of
> > > > > "highest-supported L3_MASK_n register" when SDCIAE is enabled?
> > > >
> > > > No. There is no relationship in here.
> > > >
> > > > > On the resctrl interface side the documentation currently states:
> > > > >
> > > > >      "shareable_bits":
> > > > >          Bitmask of shareable resource with other executing
> > > > >          entities (e.g. I/O). User can use this when
> > > > >          setting up exclusive cache partitions. Note that
> > > > >          some platforms support devices that have their
> > > > >          own settings for cache use which can over-ride
> > > > >          these bits.
> > > > >
> > > > > Even though this was originally used to expose the content of
> > > > > Fn0000_0010_EBX_x1[L3ShareAllocMask] the intent of the content does
> > > > > seem to also apply to the "io_alloc" CBM also.
> > > >
> > > > It says "shared by other system functions or used for some other purpose
> > > > not under the control of the PQOS feature set".
> > >
> > > This is a quote from the AMD spec, not the resctrl user interface documentation.
> > >
> > > Please consider this from resctrl user interface perspective.
> > >
> > > > "io_alloc" is PQOS feature set. I feel it should not affect "shareable_bits".
> > >
> > > When I read the resctrl user interface documentation for "shareable_bits" it
> > > sounds relevant to io_alloc. The "shareable_bits" contains a bitmask "of
> > > shareable resource with other executing entities (e.g. I/O)" ... is this
> > > not exactly io_alloc?
> >
> > I agree the text is pretty generic. Actually, the whole bit mask (0xfffF) is shareable with io_alloc.
>
> I think the value of "shareable_bits" presented to user space could be the
> actual io_alloc_cbm value. Thus, not the "possible IO bitmask" but the actual

Confused little bit here. The shareable_bits is resource property.
io_alloc_cbm is domain specific value. Not sure how to display it.

> value. This seems to be the best match of what "shareable_bits" represents, which
> is the region currently used by IO devices. This partners well with the "bit_usage"
> output, for example, "X" can be used to show which portions of cache are currently
> used by both software (via schemata of resource groups) and hardware (via io_alloc_cbm).

Haven't looked at this code much. Will look into it.
 >>
> > The 'shareable_bits' is coming from CPUID Fn0000_0010_EBX_x1[L3ShareAllocMask] which is always 0 on AMD systems.
> > It will be bit odd to manipulate these value. Not sure if we have to do it.
>
> It is not clear to me what you mean with "manipulate". "shareable_bits" does currently
> come from the existing register but AMD now provides more interfaces with which this data
> can be obtained and it seems appropriate to use it.

Ok.

Thanks
Babu