[PATCH V7 0/7] x86/intel_rdt: Intel Cache Allocation support

From: Vikas Shivappa
Date: Mon May 11 2015 - 15:04:51 EST

This patch adds a cgroup subsystem to support the new Cache Allocation
feature found in future Intel Xeon Intel processors. Cache Allocation is
a sub-feature with in Resource Director Technology(RDT) feature. RDT
provides support to control sharing of platform resources like L3

Cache Allocation Technology provides a way for the Software (OS/VMM) to
restrict L3 cache allocation to a defined 'subset' of cache which may be
overlapping with other 'subsets'. This feature is used when allocating
a line in cache ie when pulling new data into the cache. The
programming of the h/w is done via programming MSRs. The patch series
only supports to perform L3 cache allocation. However note that some of
the datastructures and framework which is common to RDT are also
implemented but its currently used only by the cache allocation

In todays new processors the number of cores is continuously increasing
which in turn increase the number of threads or workloads that can
simultaneously be run. When multi-threaded
applications run concurrently, they compete for shared
resources including L3 cache. At times, this L3 cache resource contention may
result in inefficient space utilization. For example a higher priority thread
may end up with lesser L3 cache resource or a cache sensitive app may not get
optimal cache occupancy thereby degrading the performance.
Cache Allocation kernel patch helps provides a framework for sharing L3 cache so that users
can allocate the resource according to set requirements.

More information about the feature can be found in the Intel SDM, Volume 3
section 17.15. SDM does not yet use the 'RDT' term yet and it is planned to be
changed at a later time.

*All the patches will apply on 4.1-rc0*.

Changes in V7: (Thanks to feedback from PeterZ and Matt and following
- changed lot of naming to reflect the data structures which are common
to RDT and specific to Cache allocation.
- removed all usage of 'cat'. replace with more friendly cache
- fixed lot of convention issues (whitespace, return paradigm etc)
- changed the scheduling hook for RDT to not use a inline.
- removed adding new scheduling hook and just reused the existing one
similar to perf hook.

Changes in V6:
- rebased to 4.1-rc1 which has the CMT(cache monitoring) support included.
- (Thanks to Marcelo's feedback).Fixed support for hot cpu handling for
IA32_L3_QOS MSRs. Although during deep C states the MSR need not be restored
this is needed when physically a new package is added.
-some other coding convention changes including renaming to cache_mask using a
refcnt to track the number of cgroups using a closid in clos_cbm map.
-1b cbm support for non-hsw SKUs. HSW is an exception which needs the cache
bit masks to be at least 2 bits.

Changes in v5:
- Added support to propagate the cache bit mask update for each
- Removed the cache bit mask reference in the intel_rdt structure as
there was no need for that and we already maintain a separate
closid<->cbm mapping.
- Made a few coding convention changes which include adding the
assertion while freeing the CLOSID.

Changes in V4:
- Integrated with the latest V5 CMT patches.
- Changed naming of cgroup to rdt(resource director technology) from
cat(cache allocation technology). This was done as the RDT is the
umbrella term for platform shared resources allocation. Hence in
future it would be easier to add resource allocation to the same
- Naming changes also applied to a lot of other data structures/APIs.
- Added documentation on cgroup usage for cache allocation to address
a lot of questions from various academic and industry regarding
cache allocation usage.

Changes in V3:
- Implements a common software cache for IA32_PQR_MSR
- Implements support for hsw Cache Allocation enumeration. This does not use the brand
strings like earlier version but does a probe test. The probe test is done only
on hsw family of processors
- Made a few coding convention, name changes
- Check for lock being held when ClosID manipulation happens

Changes in V2:
- Removed HSW specific enumeration changes. Plan to include it later as a
separate patch.
- Fixed the code in prep_arch_switch to be specific for x86 and removed
x86 defines.
- Fixed cbm_write to not write all 1s when a cgroup is freed.
- Fixed one possible memory leak in init.
- Changed some of manual bitmap
manipulation to use the predefined bitmap APIs to make code more readable
- Changed name in sources from cqe to cat
- Global cat enable flag changed to static_key and disabled cgroup early_init

[PATCH 1/7] x86/intel_rdt: Intel Cache Allocation detection
[PATCH 2/7] x86/intel_rdt: Adds support for Class of service management
[PATCH 3/7] x86/intel_rdt: Add support for cache bit mask management
[PATCH 4/7] x86/intel_rdt: Implement scheduling support for Intel RDT
[PATCH 5/7] x86/intel_rdt: Software Cache for IA32_PQR_MSR
[PATCH 6/7] x86/intel_rdt: Intel haswell Cache Allocation enumeration
[PATCH 7/7] x86/intel_rdt: Add Cache Allocation documentation and user
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/