Re: [PATCH v3 16/38] x86/resctrl: Move monitor init work to a resctrl init call

[Reinette Chatre]

Hi James,

On 7/1/24 11:17 AM, James Morse wrote:
> Hi Reinette,
>
> On 28/06/2024 17:47, Reinette Chatre wrote:
> > On 6/14/24 8:00 AM, James Morse wrote:
> > > rdt_get_mon_l3_config() is called from the architecture's
> > > resctrl_arch_late_init(), and initialises both architecture specific
> > > fields, such as hw_res->mon_scale and resctrl filesystem fields
> > > by calling dom_data_init().
> > >
> > > To separate the filesystem and architecture parts of resctrl, this
> > > function needs splitting up.
> > >
> > > Add resctrl_mon_resource_init() to do the filesystem specific work,
> > > and call it from resctrl_init(). This runs later, but is still before
> > > the filesystem is mounted and the rmid_ptrs[] array can be used.
>
> > > diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> > > index 7d6aebce75c1..527c0e9d7b2e 100644
> > > --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> > > +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> > > @@ -1016,12 +1016,28 @@ static void l3_mon_evt_init(struct rdt_resource *r)
> > >            list_add_tail(&mbm_local_event.list, &r->evt_list);
> > >    }
> > >    +int resctrl_mon_resource_init(void)
> >
> > (Lack of an __init is unexpected but I assume it was done since that will be removed
> > in later patch anyway?)
>
> Yup - I'll add and remove that if you find it surprising.
>
>
> > This function needs a big warning to deter anybody from considering this to
> > be the place where any and all monitor related allocations happen. It needs
> > to warn developers that only resources that can only be touched after fs mount
> > may be allocated here.
>
> I'm afraid I don't follow. Can you give an example of the scenario you are worried about?

My concern is not a scenario with current code flow but a request for informational
comments to prevent future mistakes. Specifically, as I understand the CPU online/offline
handlers can run before this function is called. Those handlers do a lot of setup, getting
resctrl and the system ready. It can be reasonable that some future action may need to touch
a new monitoring structure and with a name like resctrl_mon_resource_init() it seems appropriate
to allocate this new monitoring structure there. I am hoping that resctrl_mon_resource_init()
will have sufficient comments to deter that.

> This is called from resctrl_init(), which is called once the architecture code has done
> its setup, and reckons resctrl is something that can be supported on this platform. It
> would be safe for the limbo/overflow callbacks to start ticking after this point - but
> there is no point if the filesystem isn't mounted yet.
> Filesystem mount is triggered through rdt_get_tree(). The filesystem can't be mounted
> until resctrl_init() goes on to call register_filesystem().
> These allocations could be made later (at mount time), but they're allocated once up-front
> today.
>
>
> I've added:
> /**
>   * resctrl_mon_resource_init() - Initialise monitoring structures.

How about a more specific "Initialise monitoring structures used after filesystem mount"?

>   *
>   * Allocate and initialise the rmid_ptrs[] used for the limbo and free lists.
>   * Called once during boot after the struct rdt_resource's have been configured
>   * but before the filesystem is mounted.

Can there be a warning (please feel free to improve):
	"Only for resources used after filesystem mount. For example, do not allocate resources
	 needed by the CPU online/offline handlers since these handlers may run before this
	 function."

>   */
>
>
> Thanks,
>
> James

Reinette