Re: [PATCH v11 18/20] x86: Handle TDX erratum to reset TDX private memory during kexec() and reboot

[Huang, Kai]

On Sun, 2023-06-25 at 15:30 +0000, Huang, Kai wrote:
> On Mon, 2023-06-19 at 18:06 -0700, Dave Hansen wrote:
> > On 6/19/23 17:56, Huang, Kai wrote:
> > > Any comments to below?
> > 
> > Nothing that I haven't already said in this thread:
> > 
> > > Just use a normal old atomic_t or set_bit()/test_bit().  They have
> > > built-in memory barriers are are less likely to get botched.
> > 
> > I kinda made a point of literally suggesting "atomic_t or
> > set_bit()/test_bit()".  I even told you why: "built-in memory barriers".
> > 
> > Guess what READ/WRITE_ONCE() *don't* have.  Memory barriers.
> > 
> 
> Hi Dave,
> 
> Sorry to bring this up again.  I thought more on this topic, and I think using
> atotmic_t is only necessary if we add it right after setting up tdmr->pamt_* in
> tdmr_set_up_pamt(), because there we need both compiler barrier and CPU memory
> barrier to make sure memory order (as Kirill commented in the first reply).
> 
> However, if we add a new variable like below ...
> 
> +static bool tdx_private_mem_begin;
> +
>  /*
>   * Wrapper of __seamcall() to convert SEAMCALL leaf function error code
>   * to kernel error code.  @seamcall_ret and @out contain the SEAMCALL
> @@ -1123,6 +1125,8 @@ static int init_tdx_module(void)
>          */
>         wbinvd_on_all_cpus();
>  
> +       tdx_private_mem_begin = true;
> 
> 
> ... then we don't need any more explicit barrier, because: 1) it's not possible
> for compiler to optimize the order between setting tdmr->pamt_* and
> tdx_private_mem_begin; 2) no CPU memory barrier is needed as WBINVD is a
> serializing instruction so the wbinvd_on_all_cpus() above has already implied
> memory barrier.
> 
> Does this make sense?

Sorry please ignore this.  I missed a corner case that the kexec() can happen
when something goes wrong during module initialization and when PAMTs/TDMRs are
being freed.  We still need explicit memory barrier for this case.  I will use
atomic_t as suggested.  Thanks!