Re: [PATCH v5 2/2] x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node

[Reinette Chatre]

Hi Jarkko,

On 9/13/2021 8:04 PM, Jarkko Sakkinen wrote:
> The amount of SGX memory on the system is determined by the BIOS and it
> varies wildly between systems.  It can be from dozens of MB's on desktops
> or VM's, up to many GB's on servers.  Just like for regular memory, it is
> sometimes useful to know the amount of usable SGX memory in the system.
>
> Add an attribute for the amount of SGX memory in bytes to each NUMA
> node. The path is /sys/devices/system/node/node[0-9]*/sgx/memory_size.
>
> Signed-off-by: Jarkko Sakkinen <jarkko@xxxxxxxxxx>
>
> ---
> v5: A new patch based on the discussion at
>      https://lore.kernel.org/linux-sgx/3a7cab4115b4f902f3509ad8652e616b91703e1d.camel@xxxxxxxxxx/T/#t
> ---
>   Documentation/x86/sgx.rst      | 14 ++++++
>   arch/x86/kernel/cpu/sgx/main.c | 90 ++++++++++++++++++++++++++++++++++
>   arch/x86/kernel/cpu/sgx/sgx.h  |  2 +
>   3 files changed, 106 insertions(+)


...

> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index a6e313f1a82d..c43b5a0120c1 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -717,6 +717,7 @@ static bool __init sgx_page_cache_init(void)
>   		}
>   
>   		sgx_epc_sections[i].node =  &sgx_numa_nodes[nid];
> +		sgx_numa_nodes[nid].size += size;
>   
>   		sgx_nr_epc_sections++;
>   	}

The above memory seems to be uninitialized at the time it is incremented.

I tried this out on a system that reports the following:

$ dmesg | grep EPC
[    7.252838] sgx: EPC section 0x1000c00000-0x107f7fffff
[    7.256921] sgx: EPC section 0x2000c00000-0x207fffffff

It shows unexpectedly large values:
$ cat /sys/devices/system/node/node*/sgx/memory_size
12421486739271732874
16308428754864105707

System reported sane values after adding this fixup:

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 3380390cc052..d73bbfbfc05d 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -621,7 +621,7 @@ static bool __init sgx_page_cache_init(void)
 	int nid;
 	int i;

-	sgx_numa_nodes = kmalloc_array(num_possible_nodes(), 
sizeof(*sgx_numa_nodes), GFP_KERNEL);
+	sgx_numa_nodes = kcalloc(num_possible_nodes(), 
sizeof(*sgx_numa_nodes), GFP_KERNEL);
 	if (!sgx_numa_nodes)
 		return false;


After fixup:
$ cat /sys/devices/system/node/node*/sgx/memory_size
2126512128
2134900736


Reinette