[PATCH 0/4] sched/cache: Fixes for cache aware scheduling

[Tim Chen]

Hi all,

Cache aware scheduling went in for v7.2 and people have found a few
things wrong with it since.  We collect the fixes in this series
so it is easier to track. Two keep tasks from
being stranded outside, or yanked away from, their preferred LLC; two
fix a use after free.  Patches 1-2 stand alone, 3 and 4 go together.

Patch 1: alb_break_llc() compares nr_pref_llc_running with
cfs.h_nr_runnable, but those count different sets - one follows queued
tasks, the other drops delay-dequeued ones.  With DELAY_DEQUEUE the
equality stops holding and active balance pulls a task off its preferred
LLC.  So fix the counter.  Reported by Zhan Xusheng:
https://lore.kernel.org/lkml/20260827135000.735138-1-zhanxusheng@xxxxxxxxxx/

Patch 2 (Lu Wang): the stopper doing active load balance builds a fresh
lb_env that doesn't inherit migration_type, so can_migrate_task() can
move a task *out* of its preferred LLC.  A new LBF_ACTIVE_LB_LLC flag
and picking the stopper callback at kick time keep the intent; passing
migration_type through the stopper would muddy delayed dequeue.  v4:
https://lore.kernel.org/lkml/20260903020656.3793626-1-wanglu.priv@xxxxxxxxx/

Patches 3-4 are for the use after free Hyunwoo Kim caught with KASAN:
https://lore.kernel.org/lkml/apPb-Dr4nPYuHQOK@v4bel/
account_mm_sched() reaches the stats via p->mm->sc_stat, but a task can
be switching mm on one CPU while another is inside account_mm_sched(),
so the mm and the stats inside it can go away underneath.  Locking the
rq in the mm free path felt like the wrong trade, so patch 4 pulls
sched_cache_stat out of mm_struct into a refcounted, RCU freed
sched_cache_group - just moving code - and patch 5 does the real fix:
each task takes its own reference (copy_mm(), exec_mmap(), dropped in
exit_mm()), so the group outlives any mm switch.  Same Fixes: tag and
Hyunwoo's Tested-by on both; they want to go in together.

Nice side effect: the group no longer follows the address space,
so a user defined group, or cgroup or numa_group could own it later.
These are also the grouping by prctl RFC's first two patches, sent here
so the fix isn't held up by that discussion.

BTW, there are two other issues in discussion currently and need
a bit more work:
1. Incorrect donor context being passed to task_tick_cache().  
https://lore.kernel.org/lkml/20260909092901.2989564-1-sh_def@xxxxxxx/
It is currently under discussion and is not included in this series.
2. Cache aware scheduling interfering with ITMT.
https://lore.kernel.org/lkml/20260810033742.1688718-1-yu.c.chen@xxxxxxxxx/
https://lore.kernel.org/lkml/2fe2c681-b748-41fa-8b56-1169c86cefbc@xxxxxxxxx/ 

Applies on sched/urgent branch. 

Tim Chen and Chen Yu

Lu Wang (1):
  sched/cache: Honor migrate_llc_task semantics in active load balance

Tim Chen (3):
  sched/cache: Keep nr_pref_llc_running in the runnable domain
  sched/cache: Decouple sched_cache_group from mm
  sched/cache: Introduce task_struct->sched_cache_grp

 fs/exec.c                    |  14 ++
 include/linux/mm_types.h     |  15 +-
 include/linux/sched.h        |  11 +-
 kernel/exit.c                |  28 +++-
 kernel/fork.c                |  23 +++
 kernel/sched/build_utility.c |   4 +
 kernel/sched/cache_sched.c   |  39 +++++
 kernel/sched/fair.c          | 297 ++++++++++++++++++++++++++---------
 kernel/sched/sched.h         |   3 +
 9 files changed, 340 insertions(+), 94 deletions(-)
 create mode 100644 kernel/sched/cache_sched.c

-- 
2.32.0