Re: [PATCH-cgroup v2] cgroup: Move rcu_head up near the top of cgroup_root

[Waiman Long]

On 12/7/23 11:46, Michal Koutný wrote:
> On Thu, Dec 07, 2023 at 08:46:14AM -0500, Waiman Long <longman@xxxxxxxxxx> wrote:
> > Commit 77070eeb8821 ("cgroup: Avoid false cacheline sharing of read
> > mostly rstat_cpu") happens to be the last straw that breaks it.
> FTR, when I build kernel from that commit, I can see
>
> > struct cgroup_root {
> > 	struct kernfs_root *       kf_root;              /*     0     8 */
> > 	unsigned int               subsys_mask;          /*     8     4 */
> > 	int                        hierarchy_id;         /*    12     4 */
> >
> > 	/* XXX 48 bytes hole, try to pack */
> >
> > 	/* --- cacheline 1 boundary (64 bytes) --- */
> > 	struct cgroup              cgrp __attribute__((__aligned__(64))); /*    64  2368 */
> >
> > 	/* XXX last struct has 8 bytes of padding */
> >
> > 	/* --- cacheline 38 boundary (2432 bytes) --- */
> > 	struct cgroup *            cgrp_ancestor_storage; /*  2432     8 */
> > 	atomic_t                   nr_cgrps;             /*  2440     4 */
> >
> > 	/* XXX 4 bytes hole, try to pack */
> >
> > 	struct list_head           root_list;            /*  2448    16 */
> > 	struct callback_head       rcu __attribute__((__aligned__(8))); /*  2464    16 */
> > 	unsigned int               flags;                /*  2480     4 */
> > 	char                       release_agent_path[4096]; /*  2484  4096 */
> > 	/* --- cacheline 102 boundary (6528 bytes) was 52 bytes ago --- */
> > 	char                       name[64];             /*  6580    64 */
> >
> > 	/* size: 6656, cachelines: 104, members: 11 */
> > 	/* sum members: 6592, holes: 2, sum holes: 52 */
> > 	/* padding: 12 */
> > 	/* paddings: 1, sum paddings: 8 */
> > 	/* forced alignments: 2, forced holes: 1, sum forced holes: 48 */
> > } __attribute__((__aligned__(64)));
> 2480 has still quite a reserve below 4096. (I can't see an CONFIG_*
> affecting this.)
>
> Perhaps, I missed something from the linux-next merging thread?

CONFIG_LOCKDEP and some other debug configs are enabled with 
allmodconfig. This can greatly increase the size of some of the 
structures. I am not able to use pahole due to missing BTF info so I 
don't the exact size. However, I can reproduce the build failure and the 
patch is  able to fix it.

Cheers,
Longman