Re: [PATCH 0/2] jbd2: bound j_list_lock hold time in the checkpoint shrinker

[Max Kellermann]

On Mon, Jul 13, 2026 at 2:53 PM Zhang Yi <yi.zhang@xxxxxxxxxxxxxxx> wrote:
> I'm a bit curious about which kernel version and CONFIG you
> encountered this issue on. IIRC, after commits [1] and [2], the
> spin_needbreak() in jbd2_journal_shrink_checkpoint_list() should
> always be effective (unless the hardware does not support it, i.e.,
> ARCH_NO_PREEMPT).

We run 6.18.38, and preemption is disabled. Therefore,
spin_needbreak() is a no-op.
(We used to always run the latest stable, but got burned by way too
many critical Ceph/netfs regressions and settled on the LTS.)

But even with preemption enabled, this would likely be a problem,
though a much smaller one.

> Moreover, the number of buffer_heads in the
> checkpoint list of a single transaction should not be large enough
> to trigger such soft lockups. So I suspect this issue should be
> unlikely to occur on the latest kernel releases.

Unfortunately, I don't have any numbers for you, because this problem
occurs only every other week, but we have been haunted by this for a
long time.

The setup is a 7 TB ext4 partition dedicated to fscache/Ceph. It's
always full, and culling is running quite often.

For culling, we don't use cachefilesd but our own reimplementation
(https://github.com/CM4all/cash) because cachefilesd scales
quadratically and never makes any progress while burning CPU cycles
and I/O forever; ours is massively parallel thanks to io_uring. That
puts a lot of pressure on the ext4 filesystem, and a cgroup
memory.high setting means it's always under memory pressure (or else
the kernel will accumulate many gigabytes of RAM usage very quickly
during the filesystem scan that is necessary for culling). Which means
the shrinker is running all the time. Plus the usual kernel-initiated
I/O on that partition.
Every time this problem occurred, fscache culling was active.

-- 
Max Kellermann
Principal Architect
Hosting Technology

cm4all | Im Mediapark 6a | 50670 Köln | Germany
General information about the company can be found here:
https://www.cm4all.com/impressum
A member of the IONOS Group