Re: [PATCH 2/2] x86/sme: Mark the code as __head in mem_encrypt_identity.c

[Hou Wenlong]

On Wed, Oct 18, 2023 at 06:20:15PM +0800, Ingo Molnar wrote:
> 
> * Hou Wenlong <houwenlong.hwl@xxxxxxxxxxxx> wrote:
> 
> > On Tue, Oct 17, 2023 at 08:52:46PM +0800, Ingo Molnar wrote:
> > > 
> > > * Hou Wenlong <houwenlong.hwl@xxxxxxxxxxxx> wrote:
> > > 
> > > > The functions sme_enable() and sme_encrypt_kernel() are only called by
> > > > the head code which runs in identity virtual address. Therefore, it's
> > > > better to mark them as __head as well.
> > > > 
> > > > Signed-off-by: Hou Wenlong <houwenlong.hwl@xxxxxxxxxxxx>
> > > > ---
> > > >  arch/x86/include/asm/mem_encrypt.h |  8 ++++----
> > > >  arch/x86/mm/mem_encrypt_identity.c | 27 ++++++++++++++-------------
> > > >  2 files changed, 18 insertions(+), 17 deletions(-)
> > > > 
> > > > diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> > > > index 359ada486fa9..48469e22a75e 100644
> > > > --- a/arch/x86/include/asm/mem_encrypt.h
> > > > +++ b/arch/x86/include/asm/mem_encrypt.h
> > > > @@ -46,8 +46,8 @@ void __init sme_unmap_bootdata(char *real_mode_data);
> > > >  
> > > >  void __init sme_early_init(void);
> > > >  
> > > > -void __init sme_encrypt_kernel(struct boot_params *bp);
> > > > -void __init sme_enable(struct boot_params *bp);
> > > > +void sme_encrypt_kernel(struct boot_params *bp);
> > > > +void sme_enable(struct boot_params *bp);
> > > >  
> > > >  int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size);
> > > >  int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size);
> > > > @@ -75,8 +75,8 @@ static inline void __init sme_unmap_bootdata(char *real_mode_data) { }
> > > >  
> > > >  static inline void __init sme_early_init(void) { }
> > > >  
> > > > -static inline void __init sme_encrypt_kernel(struct boot_params *bp) { }
> > > > -static inline void __init sme_enable(struct boot_params *bp) { }
> > > > +static inline void sme_encrypt_kernel(struct boot_params *bp) { }
> > > > +static inline void sme_enable(struct boot_params *bp) { }
> > > 
> > > So I think we should preserve the previous convention of marking functions 
> > > __init in the header-declaration and at the definition site as well, and do 
> > > the same with __head as well?
> > > 
> > Hi Ingo,
> > 
> > I tried to include <asm/init.h> into <asm/mem_encrypt.h> and mark the
> > function declaration as __head, but it resulted in a build failure. This
> > is because <asm/init.h> is not self-contained; the type "pgd_t" is
> > defined in <asm/pgtable_types.h>, which includes <asm/mem_encrypt.h>,
> > leading to mutual inclusion of header files. To avoid the issue of
> > complicated header file inclusion, I removed the annotation from the
> > function declaration.
> 
> The right solution at that point is to make <asm/init.h> self-contained...
>

The "pgd_t" is a typedef declaration in <asm/pgtable_types.h>, so it
cannot be forward declared. Therefore, I had to include
<asm/pgtable_types.h> into <asm/init.h> to make it self-contained.
However, <asm/pgtable_types.h> includes <asm/mem_encrypt.h>. If I
include <asm/init.h> into <asm/mem_encrypt.h> to mark functions as
__head in the header-declaration, it would result in mutual inclusion of
header files. It appears that <asm/mem_encrypt.h> is a base header that
is included in multiple headers, so adding one more header to it would
complicate things. In reality, if it is acceptable, I could move the
__head definition into <asm/mem_encrypt.h>.
 
> > Actually, initially, I noticed that the __init definition is in
> > <linux/init.h>, so I first placed the __head definition in
> > <linux/init.h> as well. However, this conflicted with the local variable
> > in the "list_next_or_null_rcu" macro in <linux/rculist.h>. Then I
> > realized that __head was only used in x86, so I made the decision to put
> > it in the architecture-specific header. Considering simplicity, I chose
> > to put the definition in <asm/init.h>. I also attempted to put the
> > definition in other headers such as <asm/boot.h> and
> > <asm/bootparam_utils.h>, and included them in <asm/mem_encrypt.h>, but
> > the build still failed.
> 
> When exporting a localized definition you should consider namespace 
> collisions - the name '__head' is way too generic, no wonder it caused 
> problems elsewhere.
> 
> I'd suggest naming it __init_head or so, but still keep it in a x86-only 
> header.
> 
> I presume keeping it all in the  separate section and widening its usage has a 
> specific purpose? Please outline that in the changelog as well.
> 

Based on my understanding, the __head section contains the early boot
code that runs at a low identity address instead of the compile-time
address. Therefore, it must use RIP-relative addressing to access
memory. This makes the __head section special. However, when it comes to
C source code, the compiler may generate absolute addressing, which can
result in boot failure. That's why the fixup_pointer() function is
introduced in head64.c. So maybe we could consider validating the memory
access instructions in this section using objtool to ensure that the
generated instructions are PC-relative. Then we should mark all the
early boot code as __head.

Thanks!

> Ie. instead of mechanical patches that try to follow existing patterns 
> cargo-cult style, this area of x86 code requires well-argued, well thought 
> out patches that show background knowledge of the area.
> 
> Thanks,
> 
> 	Ingo