Re: [linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[Mateusz Guzik]

On Fri, Nov 6, 2026 at 12:11 PM Breno Leitao <leitao@xxxxxxxxxx> wrote:
>
> On Thu, Nov 05, 2026 at 04:57:33PM +0100, Mateusz Guzik wrote:
> > On Thu, Nov 5, 2026 at 4:28 PM Breno Leitao <leitao@xxxxxxxxxx> wrote:
> >
> > The test at hand has multiple readers and writers using the same pipe,
> > with 64KB i/os.
> >
> > Prior to the patch each writer had its own private pool to draw from,
> > guaranteeing no allocation will happen under the mutex.
> >
> > With the patch the reader exit also drains the pool.
> >
> > As a result tons of allocation ends up happening under the lock again:
> >
> > -   94.88%    94.88%  bw_pipe          [kernel.kallsyms]     [k]
> > __alloc_frozen_pages_noprof
> >      94.88% __internal_syscall_cancel
> >         entry_SYSCALL_64_after_hwframe
> >         do_syscall_64
> >         ksys_write
> >         vfs_write
> >       - anon_pipe_write
> >          - 47.46% alloc_pages_noprof
> >               alloc_pages_mpol
> >               __alloc_frozen_pages_noprof
> >          - 47.42% anon_pipe_prefill_and_lock
> >               alloc_pages_noprof
> >               alloc_pages_mpol
> >               __alloc_frozen_pages_noprof
>
> Ack, this matches what I found as well. Thanks for investigating it.
>
> Because the preallocated pages are not private to the writer anymore,
> the reader flushes them while the writer waits for room in the pipe,
> and the writer then has to allocate them inside the lock.
>
> This basically loses the wins[0] from commit 212ed884a1aeef ("fs/pipe:
> pre-allocate pages outside pipe->mutex in anon_pipe_write") for this
> specific scenario.
>
> > I don't think multiple high-volume writers vs multiple high
> > volume-readers is a realistic scenario and I would not bother fixing
> > it.
>
> Agree, but I would like to spend some cycles on it.
>
> > However, should someone want to address it, I think the patch should
> > be reverted and the resulting state tweaked. The patch at hand came
> > after the mutex is released -- the kernel prior to any of this would
> > still put_page on the reader side under the lock. This bit would have
> > to get tweaked to recover the loss in that scenario (not to be
> > confused with the loss reported here).
>
> We can revert the patch, but that would bring back the two page pools
> (the on-stack prealloc and tmp_page[]), which is probably worse than
> this regression itself (IIRC it would also lose the gains for smaller
> writes (about 20% for 8-16 KiB writes IIRC ))
>

Like I said above, per my understanding the win is not specific to the
unified pool thing, but merely to the fact that the reader would put
pages *after* releasing the lock. this can be done in either approach.

> That said, these are the items I am planning to spend some time one:
>
> 1) Reclaiming the pooled pages from a shrinker, as suggested in [1]
>

The more I look at this the more the extra memory usage bothers me --
even with shrinker support there is just plain waste until it runs and
it is going to cost elsewhere. If this was "free" performance I would
have no comments.

Makes me wonder if there is any practical win to pursuing it in the first place.

But here is an idea just to ponder: what if the threads using the pipe
were guaranteed to trim it at some point, even if it remains open and
unused?

Currently there is the notion of task_work, which allows for arbitrary
callbacks to execute on the user<->kernel boundary. As implemented it
is rather expensive (atomics) to use and always flushes the entire
work queue.

But imagine the mechanism got extended such that there is a separate
queue where only current can add stuff (so it is cheap). Getting to
the code could be gated with the same branch which tells the kernel
about task_work, so no extra overhead in syscall handling as long as
you don't have anything queued up.

With this in place pipe code seeing a big buffer could add a callback,
which would do some magic to decide whether to trim the pool and by
what amount. Just for the sake of argument, here is an example: 1/16
chance it will do anything to begin with. if it chooses to do
something, it checks if there is a "trimmer was here" bit. If no, sets
the bit and leaves. If yes, shrinks the pool in half. The pipe write
code would clear the bit, so a pipe with continuous usage would mostly
remain intact and an unused pipe would eventually drain to nothing
(including the currently held 2 pages).

> 2) Trimming the pool gradually on the reader side, instead of cutting it
>    down to two pages in one go. Or not trimming it while a writer is
>    in the waiting room (!?)
>

There is no indicator there is a writer pending, at best you can check
if the lock is contested. This seems like gaming the benchmark to me
though, as a real program who happened to generate data a little later
would suddenly find itself with the pool trimmed.

Granular trimming definitely makes sense, provided shrinker support is
implemented.

> 3) Keeping the writer's preallocated pages private again, while keeping
>    the unified pool (i.e. without bringing tmp_page[] back).
>
> 4) Freeing the pages the reader drops after releasing pipe->mutex,
>    instead of under it, as you suggested. This is independent of the
>    options above.
>
> 5) Longer term: copying the data into the preallocated pages before
>    taking pipe->mutex, so the writer only links them into the pipe
>    under the lock. This doesn't seem trivial, but it would make this
>    whole pre-allocated buffers go away
>

I think this is sounds awfully complicated in face of multiple writers.

> 6) Any other idea?

The SMAP stuff is the primary problem right now. To reiterate: every
single buffer read/write takes a SMAP trip, which is very expensive
even on modern uarchs. Refactoring this into i/o under one SMAP round
would provide a visible speed up. It also happens to not pose a
tradeoff in terms of memory usage and would be my first avenue of
attack if I was working on this.

All that said, one reader vs one writer scenario definitely makes
sense to optimize.

Apart from that, highly parallel i/o on one pipe is imo the wrong
thing to look at in the first place. What is more likely to occur in
the real world is the user having separate processes operating on
distinct pipes, but this was not benchmarked at all. Another case to
look at is one reader vs one writer, but all the tuples have unique
uids.