Re: [PATCH v6] mm: retry page faults once under the per-VMA lock

[Hongru Zhang]

On Tue, Sep 22, 2026 at 1:54 AM Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx> wrote:
> So 2 things the patch needs to answer, more or less right away:
>
> 1. What are you actually optimising for?
>
> 2. Where does the hurt happen?
>
> Your patch does neither, then goes on to arm wave away all the complexity
> and presents a 'simple' solution that I feel brushes a lot under the rug.
>
> Answer to 1 is 'cold app startup time for a specific set of apps'
> 2 is filemap_fault() AFAICT.
>
> You should definitely be stating this upfront.

Thank you for the detailed review, Lorenzo, and thank you Matthew for
the follow-up. The analysis of the existing VM_FAULT_RETRY semantics is
very helpful.

Let us answer these two questions directly.

The problem we are addressing is `mmap_lock` contention caused by the
per-VMA lock falling back to `mmap_lock` on `VM_FAULT_RETRY`.

The core pattern is both `mmap_lock` waiters blocking progress and a
priority inversion: page fault threads hold the read lock while
blocking on I/O (`filemap_fault`, `do_swap_page`); a writer
(`mmap`/`munmap`/`mprotect` from the linker or runtime) waits for the
write lock; and all subsequent readers, which may still require
`mmap_lock` in the page fault retry path, including the UI main thread,
queue behind the writer. For the priority inversion issue, the main
thread can stall until the entire chain is resolved.


This contention has different consequences depending on context:

- During app cold startup, it slows down launch time.
- During runtime, on a 16.67ms vsync budget, it directly causes dropped
  frames — one of the most user-visible performance issues on Android.
- It also degrades throughput for any concurrent page fault workload
  under write pressure (as the benchmarks below show).

> Your perf numbers look very artificial - you create situations where you
> KNOW from the code you'll get mmap lock contention, then have a lot of
> threads contend and gosh darn it, well the contention is really high! :)
>
> I mean - I feel like most of your perf numbers amount to 'contend locks
> gets us lock contention' - and are thus useless really?

We would not say the microbenchmarks are purely artificial. The filemap
benchmark [1] specifically mimics the scenario Matthew described earlier,
where pages can be reclaimed while a page fault is being retried,
resulting in another round of I/O. Under concurrent `mmap` write
pressure, the VMA lock fallback to `mmap_lock` turns this into severe
`mmap_lock` contention. In fact, waiting on `mmap_lock` for an extended
period during the page fault retry can give reclaim a sufficient window
to reclaim the page again.

The swap benchmark [2] exercises the `do_swap_page` path under the same
conditions. These are not purely arbitrary contention generators, but
simplified models of the real problem. While the effect may be amplified
compared with some real workloads with less contention, they reproduce
the exact fault paths where the problem occurs in a simpler way.

That said, we agree the real-world impact is what matters, and we have
more data to present.


> I think the bit that _matters_ is what you're actually after which is stuff
> like:
>
> > Tencent Video cold app launch time
> >
> > +-----------+----------+----------+--------+
> > | Statistic |  Vanilla |  Patched | Change |
> > +-----------+----------+----------+--------+
> > |    Mean   | 1,907 ms | 1,840 ms |  -3.5% |
> > +-----------+----------+----------+--------+
> > |  Maximum  | 3,023 ms | 2,851 ms |  -5.7% |
> > +-----------+----------+----------+--------+
>
> Which is not earth-shattering to me.

The impact of `mmap_lock` contention on cold startup time can vary
significantly across apps, depending on their I/O volume, number of
concurrent threads, and library loading patterns. For example, Vivo
independently tested Douyin (TikTok China) cold launch on SM8975 and
observed a 15.5% improvement (666 → 563 ms, average of 100 runs) [3].
Douyin's heavier concurrent memory activity makes it more sensitive to
this contention.


> Now, while the patch is _small_ it takes all of the above, all of the
> complexity, all of the bigger questions about how to solve this and
> compresses it down into 'just retry the VMA lock'.
>
> But I worry a LOT more that it's just a HACK (and I know Barry was nice
> about the idea and I appreciate it but I have to be honest).
>
> Instead of dealing with any of the above, we just leave the mess in place +
> just retry the operation under VMA because we happen to know, for this ONE
> workload, it works out better.

This is not a hack for one workload. The priority-inversion pattern
described above is inherent to any multi-threaded process with concurrent
page faults and mmap operations — which covers essentially all modern
Android apps (zygote fork + GC + JIT + IO threads).

> ALSO. It seem we are subtly changing how the tried flag stuff works and
> that really needs to be considered carefully.
>
> And FINALLY if we DO do that, the patch message MUST be honest about the
> goals MUST state EXACTLY what it's trying to optimise, it MUST clearly
> state that in a comment or whatever where the code is.
>
> It must NOT wave its arms and pretend that it's some general solution for
> fault retries the way it does now. It feels like it's hiding what it's
> really intended for at that just adds yet more confusion to this mess.

To supplement the cold startup data we presented earlier, here is frame
drop data that we did not have in previous versions. We traced mmap_lock
contention on the UI main thread across 17 popular Android apps during
typical scroll/swipe interactions, 10 runs each:

Overall:

  Vanilla:  722 contention events, 12,315 ms total contention time
  Patched:  235 contention events,  4,284 ms total contention time
  Change:   -67.5% events, -65.2% total time

Per-app breakdown (sorted by time reduction):

  App              Vanilla(cnt/ms)    Patched(cnt/ms)    Count     Time
  -----------------------------------------------------------------------
  NetEase Music    50 /   521 ms      0 /     0 ms      -100.0%  -100.0%
  Meituan          13 /    81 ms      0 /     0 ms      -100.0%  -100.0%
  Tencent Video   158 / 2,452 ms     17 /   177 ms       -89.2%   -92.8%
  Toutiao          30 /   289 ms      5 /    24 ms       -83.3%   -91.6%
  Douyin           41 /   572 ms      6 /    95 ms       -85.4%   -83.4%
  Douyin Lite      28 /   565 ms     10 /   118 ms       -64.3%   -79.1%
  Pipixia          18 /   533 ms      6 /   127 ms       -66.7%   -76.2%
  Amap             10 /    46 ms      3 /    15 ms       -70.0%   -67.7%
  JD.com           80 / 3,145 ms     32 / 1,172 ms       -60.0%   -62.7%
  Kuaishou Lite    75 /   978 ms     11 /   366 ms       -85.3%   -62.6%
  WeChat            1 /    17 ms      2 /     6 ms      +100.0%   -61.5%
  Alipay           14 /   274 ms      8 /   108 ms       -42.9%   -60.5%
  Douyu             2 /    16 ms      2 /     8 ms        +0.0%   -46.4%
  Kuaishou         55 /   686 ms     22 /   432 ms       -60.0%   -37.0%
  Baidu Tieba      57 / 1,441 ms     42 / 1,010 ms       -26.3%   -29.9%
  Weibo            52 /   445 ms     36 /   382 ms       -30.8%   -14.2%
  Bilibili         38 /   257 ms     33 /   243 ms       -13.2%    -5.2%

All 17 apps show reduction in main-thread contention time, with 12
of them seeing >60% reduction. These are not synthetic benchmarks —
they are real apps running on real devices.

I have also attached a Perfetto diagram showing how the main thread
waiting for `mmap_lock` during a page fault retry can lead to missed
frames.


[1] https://gist.github.com/zhr250/c36c2c54d9351df37e12fd072d4926ef
[2] https://gist.github.com/zhr250/218ffe693f842346b56434483127422c
[3] https://lore.kernel.org/linux-mm/20260915130544.3774446-1-xueyuan.chen@xxxxxxxx/

Thanks,
Hongru

Attachment: perfetto.jpg
Description: JPEG image