Re: [PATCH] kernfs: don't hold kernfs_rwsem across notification delivery

From: Tejun Heo

Date: Thu Sep 10 2026 - 17:12:48 EST


On Wed, Sep 09, 2026 at 09:54:06PM -0700, Shakeel Butt wrote:
> Recently at Meta we noticed that on multi-tenant systems, a workload
> under extreme memory pressure negatively impacts unrelated workloads and
> system daemons like below [1] and fb-oomd [2]. Mainly we observed that
> system daemons were stuck on kernfs_rwsem in cgroup-related interfaces
> for long periods of time.
>
> On further inspection, we found that the workload under pressure had
> registered inotify watches on its memory.events and was continuously
> receiving limit-hitting notifications. The cgroup notification is done in
> kernfs_notify_workfn(), which holds kernfs_supers_rwsem and kernfs_rwsem
> and calls fsnotify(), which allocates the event with
> kmalloc(GFP_KERNEL_ACCOUNT|__GFP_RETRY_MAYFAIL) charged to the watching
> cgroup, which is already under memory pressure. So the notify worker
> ended up in memory reclaim of a cgroup already under pressure, and itself
> kept triggering the limit notifications.
>
> This notification path takes the locks in read mode, but a waiting writer
> blocks all future readers. That is exactly what is happening in the Meta
> fleet, and it leaves daemons that are critical to the reliability of the
> system stuck for long periods of time.
>
> Commit 400188ae361a ("kernfs: Acquire kernfs_rwsem in
> kernfs_notify_workfn().") added kernfs_rwsem just to safely read
> kernfs_node::name. Let's sample the name once before the loop and drop
> the lock. kernfs_supers_rwsem still covers the list, and the removal
> paths take that one for reading too, so a stalled worker can still hold
> a removal up once a mount or unmount is queued behind it.
>
> Two things change for a watcher. A name longer than NAME_MAX now gives
> an event with no name, where before it gave one with the full name;
> fsnotify() takes the name as optional, so the event still arrives. And
> the name and the parent are no longer sampled under one lock, so a rename
> between the two would name a file against the directory it moved to.
> That cannot happen today: kernfs_notify() rejects anything that is not a
> file, and no caller of kernfs_rename_ns() renames one.
>
> [1] https://github.com/facebookincubator/below
> [2] https://github.com/facebookincubator/oomd
>
> Fixes: 400188ae361a ("kernfs: Acquire kernfs_rwsem in kernfs_notify_workfn().")
> Assisted-by: LLM
> Signed-off-by: Shakeel Butt <shakeel.butt@xxxxxxxxx>

Acked-by: Tejun Heo <tj@xxxxxxxxxx>

Thanks.

--
tejun