Re: [PATCH] ACPI/HMAT: Fix the parsing of Cache Associativity and Write Policy

[Dan Williams]

On Tue, Dec 10, 2019 at 7:05 PM Tao Xu <tao3.xu@xxxxxxxxx> wrote:
>
>
> On 12/10/19 9:18 PM, Tao Xu wrote:
> > On 12/10/2019 4:27 PM, Rafael J. Wysocki wrote:
> >> On Tue, Dec 10, 2019 at 9:19 AM Tao Xu <tao3.xu@xxxxxxxxx> wrote:
> >>>
> >>> On 12/10/2019 4:06 PM, Rafael J. Wysocki wrote:
> >>>> On Tue, Dec 10, 2019 at 2:04 AM Tao Xu <tao3.xu@xxxxxxxxx> wrote:
> >>>>>
> >>>>> On 12/9/2019 6:01 PM, Rafael J. Wysocki wrote:
> >>>>>> On Mon, Dec 2, 2019 at 8:03 AM Tao Xu <tao3.xu@xxxxxxxxx> wrote:
> >>>>>>>
> >>>>>>> In chapter 5.2.27.5, Table 5-147: Field "Cache Attributes" of
> >>>>>>> ACPI 6.3 spec: 0 is "None", 1 is "Direct Mapped", 2 is "Complex
> >>>>>>> Cache
> >>>>>>> Indexing" for Cache Associativity; 0 is "None", 1 is "Write Back",
> >>>>>>> 2 is "Write Through" for Write Policy.
> >>>>>>
> >>>>>> Well, I'm not sure what the connection between the above statement,
> >>>>>> which is correct AFAICS, and the changes made by the patch is.
> >>>>>>
> >>>>>> Is that the *_OTHER symbol names are confusing or something deeper?
> >>>>>>
> >>>>>
> >>>>> Because in include/acpi/actbl1.h:
> >>>>>
> >>>>> #define ACPI_HMAT_CA_NONE                     (0)
> >>>>>
> >>>>> ACPI_HMAT_CA_NONE is 0, but in include/linux/node.h:
> >>>>>
> >>>>>       enum cache_indexing {
> >>>>>              NODE_CACHE_DIRECT_MAP,
> >>>>>              NODE_CACHE_INDEXED,
> >>>>>              NODE_CACHE_OTHER,
> >>>>>       };
> >>>>> NODE_CACHE_OTHER is 2, and for otner enum:
> >>>>>
> >>>>>             case ACPI_HMAT_CA_DIRECT_MAPPED:
> >>>>>                     tcache->cache_attrs.indexing =
> >>>>> NODE_CACHE_DIRECT_MAP;
> >>>>>                     break;
> >>>>>             case ACPI_HMAT_CA_COMPLEX_CACHE_INDEXING:
> >>>>>                     tcache->cache_attrs.indexing = NODE_CACHE_INDEXED;
> >>>>>                     break;
> >>>>> in include/acpi/actbl1.h:
> >>>>>
> >>>>>     #define ACPI_HMAT_CA_DIRECT_MAPPED            (1)
> >>>>>     #define ACPI_HMAT_CA_COMPLEX_CACHE_INDEXING   (2)
> >>>>>
> >>>>> but in include/linux/node.h:
> >>>>>
> >>>>> NODE_CACHE_DIRECT_MAP is 0, NODE_CACHE_INDEXED is 1. This is
> >>>>> incorrect.
> >>>>
> >>>> Why is it incorrect?
> >>>
> >>> Sorry I paste the wrong pre-define.
> >>>
> >>> This is the incorrect line:
> >>>
> >>> case ACPI_HMAT_CA_DIRECT_MAPPED:
> >>> tcache->cache_attrs.indexing = NODE_CACHE_DIRECT_MAP;
> >>>
> >>> ACPI_HMAT_CA_DIRECT_MAPPED is 1, NODE_CACHE_DIRECT_MAP is 0. That means
> >>> if HMAT table input 1 for cache_attrs.indexing, kernel store 0 in
> >>> cache_attrs.indexing. But in ACPI 6.3, 0 means "None". So for the whole
> >>> switch codes:
> >>
> >> This is a mapping between the ACPI-defined values and the generic ones
> >> defined in the kernel.  There is not rule I know of by which they must
> >> be the same numbers.  Or is there such a rule which I'm missing?
> >>
> >> As long as cache_attrs.indexing is used consistently going forward,
> >> the difference between the ACPI-defined numbers and its values
> >> shouldn't matter, should it?
> >>
> > Yes, it will not influence the ACPI HMAT tables. Only influence is the
> > sysfs, as in
> > https://www.kernel.org/doc/html/latest/admin-guide/mm/numaperf.html:
> >
> > # tree sys/devices/system/node/node0/memory_side_cache/
> > /sys/devices/system/node/node0/memory_side_cache/
> > |-- index1
> > |   |-- indexing
> > |   |-- line_size
> > |   |-- size
> > |   `-- write_policy
> >
> > indexing is parsed in this file, so it can be read by user-space.
> > Although now there is no user-space tool use this information to do some
> > thing. But I am wondering if it is used in the future, someone use it to
> > show the memory side cache information to user or use it to do
> > performance turning.
>
> I finish a test using emulated ACPI HMAT from QEMU
> (branch:hmat https://github.com/taoxu916/qemu.git)
>
> And I get the kernel log and sysfs output:
> [    0.954288] HMAT: Cache: Domain:0 Size:20480 Attrs:00081111 SMBIOS
> Handles:0
> [    0.954835] HMAT: Cache: Domain:1 Size:15360 Attrs:00081111 SMBIOS
> Handles:0
>
> /sys/devices/system/node/node0/memory_side_cache/index1 # cat indexing
> 0
> /sys/devices/system/node/node0/memory_side_cache/index1 # cat write_policy
> 0
>
> Note that 'Attrs' is printed using %x, so we can get:
> (attrs & ACPI_HMAT_CACHE_ASSOCIATIVITY) >> 8 = 1,
> (attrs & ACPI_HMAT_WRITE_POLICY) >> 12       = 1
>
> but we get 0 in sysfs, so if user or software read this information and
> read the ACPI 6.3 spec, will think there is 'none' for Cache
> Associativity or Write Policy.

The sysfs interface is not meant to reflect the ACPI values. This
sysfs information may be populated by another platform firmware
(non-ACPI). I would have preferred that these files use text values
rather than numbers. However, at least the ABI documentation gives the
expected translation:

What:           /sys/devices/system/node/nodeX/memory_side_cache/indexY/indexing
Date:           December 2018
Contact:        Keith Busch <keith.busch@xxxxxxxxx>
Description:
                The caches associativity indexing: 0 for direct mapped,
                non-zero if indexed.

What:
/sys/devices/system/node/nodeX/memory_side_cache/indexY/write_policy
Date:           December 2018
Contact:        Keith Busch <keith.busch@xxxxxxxxx>
Description:
                The cache write policy: 0 for write-back, 1 for write-through,
                other or unknown.