Re: [PATCH v2 20/27] KVM: x86/mmu: Use page-track notifiers iff there are external users

[Yan Zhao]

On Thu, May 04, 2023 at 12:54:40PM -0700, Sean Christopherson wrote:
> On Wed, Mar 15, 2023, Sean Christopherson wrote:
> > On Wed, Mar 15, 2023, Yan Zhao wrote:
> > > On Fri, Mar 10, 2023 at 04:22:51PM -0800, Sean Christopherson wrote:
> > > > Disable the page-track notifier code at compile time if there are no
> > > > external users, i.e. if CONFIG_KVM_EXTERNAL_WRITE_TRACKING=n.  KVM itself
> > > > now hooks emulated writes directly instead of relying on the page-track
> > > > mechanism.
> > > > 
> > > > Signed-off-by: Sean Christopherson <seanjc@xxxxxxxxxx>
> > > > ---
> > > >  arch/x86/include/asm/kvm_host.h       |  2 ++
> > > >  arch/x86/include/asm/kvm_page_track.h |  2 ++
> > > >  arch/x86/kvm/mmu/page_track.c         |  9 ++++-----
> > > >  arch/x86/kvm/mmu/page_track.h         | 29 +++++++++++++++++++++++----
> > > >  4 files changed, 33 insertions(+), 9 deletions(-)
> > > > 
> > > > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> > > > index 1a4225237564..a3423711e403 100644
> > > > --- a/arch/x86/include/asm/kvm_host.h
> > > > +++ b/arch/x86/include/asm/kvm_host.h
> > > > @@ -1265,7 +1265,9 @@ struct kvm_arch {
> > > >  	 * create an NX huge page (without hanging the guest).
> > > >  	 */
> > > >  	struct list_head possible_nx_huge_pages;
> > > > +#ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING
> > > >  	struct kvm_page_track_notifier_head track_notifier_head;
> > > > +#endif
> > > >  	/*
> > > >  	 * Protects marking pages unsync during page faults, as TDP MMU page
> > > >  	 * faults only take mmu_lock for read.  For simplicity, the unsync
> > > > diff --git a/arch/x86/include/asm/kvm_page_track.h b/arch/x86/include/asm/kvm_page_track.h
> > > > index deece45936a5..53c2adb25a07 100644
> > > > --- a/arch/x86/include/asm/kvm_page_track.h
> > > > +++ b/arch/x86/include/asm/kvm_page_track.h
> > > The "#ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING" can be moved to the
> > > front of this file?
> > > All the structures are only exposed for external users now.
> > 
> > Huh.  I've no idea why I didn't do that.  IIRC, the entire reason past me wrapped
> > track_notifier_head in an #ifdef was to allow this change in kvm_page_track.h.
> > 
> > I'll do this in the next version unless I discover an edge case I'm overlooking.
> 
> Ah, deja vu.  I tried this first time around, and got yelled at by the kernel test
> robot.  Unsuprisingly, my second attempt yielded the same result :-)
> 
>   HDRTEST drivers/gpu/drm/i915/gvt/gvt.h
> In file included from <command-line>:
> gpu/drivers/gpu/drm/i915/gvt/gvt.h:236:45: error: field ‘track_node’ has incomplete type
>   236 |         struct kvm_page_track_notifier_node track_node;
>       |                                             ^~~~~~~~~~
> 
> The problem is direct header inclusion.  Nothing in the kernel includes gvt.h
> when CONFIG_DRM_I915_GVT=n, but the header include guard tests include headers
> directly on the command line.  I think I'll define a "stub" specifically to play
> nice with this sort of testing.  Guarding the guts of gvt.h with CONFIG_DRM_I915_GVT
> would just propagate the problem, and guarding the node definition in "struct
> intel_vgpu" would be confusing since the guard would be dead code for all intents
> and purposes.
> 
> The obvious alternative would be to leave kvm_page_track_notifier_node outside of
> the #ifdef, but I really want to bury kvm_page_track_notifier_head for KVM's sake,
> and having "head" buried but not "node" would also be weird and confusing.
> 
> diff --git a/arch/x86/include/asm/kvm_page_track.h b/arch/x86/include/asm/kvm_page_track.h
> index 33f087437209..3d040741044b 100644
> --- a/arch/x86/include/asm/kvm_page_track.h
> +++ b/arch/x86/include/asm/kvm_page_track.h
> @@ -51,6 +51,12 @@ void kvm_page_track_unregister_notifier(struct kvm *kvm,
>  
>  int kvm_write_track_add_gfn(struct kvm *kvm, gfn_t gfn);
>  int kvm_write_track_remove_gfn(struct kvm *kvm, gfn_t gfn);
> +#else
> +/*
> + * Allow defining a node in a structure even if page tracking is disabled, e.g.
> + * to play nice with testing headers via direct inclusion from the command line.
> + */
> +struct kvm_page_track_notifier_node {};
>  #endif /* CONFIG_KVM_EXTERNAL_WRITE_TRACKING */
>  
>  #endif
>
Or check CONFIG_KVM_EXTERNAL_WRITE_TRACKING in gvt.h ?
e.g.

diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index 53a0a42a50db..005cdc4fb66a 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -233,7 +233,9 @@ struct intel_vgpu {
        unsigned long nr_cache_entries;
        struct mutex cache_lock;

+#ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING
        struct kvm_page_track_notifier_node track_node;
+#endif
 #define NR_BKT (1 << 18)
        struct hlist_head ptable[NR_BKT];
 #undef NR_BKT

The justification is that gvt.h can be include without kvmgt, e.g. xengt
previously.
But given currently there's no such case, I'm fine with both way :)