Re: [RFC PATCH] zram: avoid preemption with CPU-based compression backends

[Bo Zhang]

On (26/08/05 19:25), Sergey Senozhatsky wrote:
> quick and dirty patch.  Just curious if this improves anything on your
> side.

Hi Sergey, Barry,

I ran an A/B of the three approaches on a real Android device
(MemTotal ~7.7GB, zram ~8GB). Same workload for all three: cycle 20
memory-heavy apps through fg/bg to force swap-out/in, 8 rounds (~550s).
The stream mutex is instrumented with a trylock-based counter; the
measurement code is identical across all three builds, so the only
difference is the patch.

                      baseline    preempt-disable   read/write split
  contended count     8,765       150               6,557
  wait_ns total       52.28 s     1.29 s            65.86 s
  wait_ns max         95.6 ms     46.2 ms           130.7 ms
  swap-out volume     ~2.86 GB    ~3.04 GB          ~2.87 GB

For the split build, contention by stream:
  read (decompress): 6,349 (96.8%)
  write (compress):    208 ( 3.2%)

Summary:
- preempt-disable: contention -98% (8765 -> 150), total wait -97.5%
  (52.3s -> 1.29s), worst case halved.
- read/write split: barely helps (-25%), total wait and worst case
  actually got worse.

The split doesn't help because 96.8% of the contention is on the read
stream alone, which is concurrent swap-in decompressions queuing on the
same per-CPU read stream while the holder gets preempted/migrated (big
-> little core), not readers vs writers. Splitting by direction doesn't
address that, and it doubles per-CPU stream memory.

So in our workload the root cause is the holder being preemptible
or migratable, which preempt-disable addresses directly.

Happy to share scripts or re-run any variation.

Thanks,
Bo