Re: [PATCH v6 1/4] block: add task-context bio completion infrastructure

[Christoph Hellwig]

On Wed, Jul 29, 2026 at 02:11:17AM -0400, Tal Zussman wrote:
> No worries, and apologies again for the delay. I had some deadlines and
> travel that kept me busy, but I have some additional results below:

Cool!

> 
> First, I investigated the workload C behavior further. The underlying
> problem is that ioends routed to the XFS completion workqueue were also
> carrying BIO_COMPLETE_IN_TASK, so unwritten completions bounced through the
> block-layer worker (plus its delay) only to be queued on
> m_unwritten_workqueue afterwards. The ioend workqueue already provides the
> task context the flag asks for, so something like this avoids the bounce:

Ah, cool - yes, we should not double-defer.  I suspect we might be
able to do something cleaner than the setting and clearing, but as I
have some work pending in that area I'm happy to look into that after
the basic series lands.

> C) 4k buffered randwrite, qd32 (~118k completions/s)
>   bandwidth (MB/s)                 649.5       655.7       643.4
>   write lat p50 (us)               215.4       214.7       216.1
>   write lat p99.9 (us)             703.1       984.4       730.5
>   context switches                20.69M      21.98M      21.44M
>   cycles                         922.5e9     838.1e9     911.3e9

This is a surprisingly larger drop.

> D) 64k buffered mixed r/w, qd32 (~11k completions/s)
>   bandwidth (MB/s)                 761.2       753.2       757.2

and this a somewhat surprising improvement.

How stable are the results?

> Again, my main concern with this approach is that 16000 (or 1000) is a
> somewhat arbitrary value that depends on disk performance. If the C p99.9
> perf is an acceptable trade-off, I think we can get rid of the delay, but
> I'd appreciate more thoughts on that and the XFS diff above.
> 
> I'll send v7 with the rest of the changes (Sashiko review, etc.) in a day or
> two, pending any additional feedback.

I'd be tempted to do delay=0 for now to get things going, and then
look into fine-tuning to see if we can come up with a good threshold
instead.

I also think in the longer run avoid the current workqueue and having
some sort of thread without the workqueue scheduling overhead would
be beneficial, as that would entirely avoid wakeups if the thread is
running.  But I'd rather get a nicely working API in first and then
fine tune it later.