Re: [PATCH RFC 00/15] mm/mglru: frequency guided promotion (MGLRU-FG) and flag cleanup

[zhaozhengzhuo]

Hi Kairui,

Thanks for the MGLRU-FG series - it is a very interesting and
well-written piece of work. I ported it locally and ran a subset of the
benchmarks from your cover letter on our test VM. The results are
positive and consistent with yours, so I wanted to share them as early
feedback. I would be happy to test the next posted revision directly
and provide a formal Tested-by tag then.

Test environment
----------------
- Host:    96 physical cores / 192 threads
- Storage: VM images on a 4 TB Netac SATA SSD (no NVMe); VM disks
           are virtio-backed image files on that SSD
- VM:      96 vCPUs, 128 GiB RAM (KVM)
- Baseline: mm-new @ a032d41a86cb
- MGLRU-FG squashed into local commit 9fc6ff682d08
- NOTE: the port additionally contains a local guard in
  folio_inc_lru_refs() for non-LRU folios, so this is not an exact
  bit-for-bit test of the posted series.

1. fio, Zipf 0.9 buffered 4 KiB random reads (all IO through the
   page cache), 40 GiB total files, 16 jobs, 16 GiB memcg,
   45 s ramp + 600 s run:

                        run 1       run 2       run 3       median
   CLRU                 797.64      796.74      800.21      797.64 MB/s
   MGLRU before         766.88      273.75(*)   769.81      766.88 MB/s
   MGLRU-FG             838.16      840.26      367.92(*)   838.16 MB/s

   Median improvement over MGLRU before: +9.30% (cover letter: 11.1%).
   The two marked runs entered an unexplained low-throughput state;
   CLRU stayed at 797-800 MB/s, and a reboot restored MGLRU-before to
   769.81 MB/s, so I kept the outliers visible rather than dropping
   them. I am investigating whether this is related to MGLRU
   history/state across hot switches.

2. MongoDB 4.0.23 / YCSB 0.17.0 workload B
   20 M records, 6 M operations, 48 threads, 16 GiB memcg, 8 GiB
   WiredTiger cache:

                        run 1       run 2       run 3       median
   CLRU                 18695.72    18673.44    18438.05    18673.44 ops/s
   MGLRU before         17517.69    16196.08    18180.28    17517.69 ops/s
   MGLRU-FG             19871.70    19721.86    19321.00    19721.86 ops/s

   MGLRU-FG improved median throughput by +12.58% (cover letter:
   10.3%). Median read latency / p95 / p99 improved by 11.35% /
   12.30% / 26.06%, respectively.
   Caveat: the master DB was loaded with w=0 and stopped too soon, so
   each run had about 22.7k READ NOT_FOUND results (~0.4% of reads).
   Every variant was cloned from the same XFS reflink master, so I
   think the relative result is still valid, but I will redo it with a
   fully settled master.

3. SQLite hot lookup + cold grep stream (sqlite-grep test, 90,000 cold
   files, 300 MiB memcg); median of run-medians for the hot lookup:

   MGLRU before:         277.18 ms
   MGLRU-FG:              13.74 ms   (-95.0%)

   Eight of ten MGLRU-FG samples were about 13-14 ms, whereas most
   MGLRU-before samples were 249-322 ms. Grep time itself changed by
   only about +1%. Compared with MGLRU before, MGLRU-FG also reduced
   file refaults by about 21%, direct scans by about 17.5%, and pgpgin
   by about 15.1%. This was the clearest reproduction of the intended
   protection against a one-shot cold stream evicting hot data.

General observations
--------------------
- No memcg OOM kills, and no kernel BUG / Oops / Call Trace / panic in
  dmesg across all completed runs.
- Confirmed that folio_inc_lru_refs() is exercised at high frequency
  under the fio workload.
- make -j48 / 3 GiB memcg / disk-swap (CLRU side only) completed in
  2h18m59.9s with 20.9M file refaults and 48.1M anon refaults, no OOM
  kill. This VM builds about 13.4k objects and many modules, so its
  absolute time is not comparable with the machine in the cover letter.
  The before/after MGLRU sides are still running; I can send the
  same-machine comparison separately.

One thing I want to flag: my local port adds a guard in
folio_inc_lru_refs() for non-LRU folios. I noticed that syzbot
reported a WARNING in folio_inc_lru_refs() on this series
("!memcg && !mem_cgroup_disabled()" in the exit_mmap path). I am not
sure whether the two are related, but if it helps, I can check whether
my guard's trigger path matches that report.

Because this was a local port plus an additional guard, I am reporting
these results without a Tested-by tag for now. I would be happy to test
the next posted revision directly and provide a formal Tested-by then.

Thanks and best regards,
zhaozhengzhuo
zhaozhengzhuo@xxxxxxxxxxxxx