Re: [RFC PATCH 00/15] io_uring: thread identity handoff for blocking inline issue

[Jens Axboe]

On 9/14/26 1:22 PM, Peter Zijlstra wrote:
> On Fri, Sep 11, 2026 at 09:40:50AM -0600, Jens Axboe wrote:
>  
>> This series issues those requests inline in blocking mode instead, and
>> only pays for the offload if the request actually blocks. But by the
>> time it blocks, the submitter is deep in the kernel with the request on
>> its stack, so the work can't be moved to another thread. What we can
>> move is the identity. If the submitting task blocks, an idle io-wq
>> worker takes over its user visible identity (tid, signal state,
>> credentials, scheduling attributes, cgroup, user register state),
>> finishes the io_uring_enter() call and returns to userspace as the
>> submitter. The original task finishes the request as an
>> io-wq worker and joins the pool. Userspace is none the wiser, hopefully,
>> the same tid came back from the syscall, it's just on a different
>> task_struct. 
> 
> I'm still struggling my way through this thing.

I don't blame you.

> So thread T1 is doing this blocking syscall. When it actually blocks,
> you hand-over the userspace identifying part to another thread T2, which
> will return to userspace as if it were T1.

Correct.

> Is this really a hand-over, or a swap? I would imagine we not have two
> threads with the same tid and all that.

It's a swap, yes we won't have two threads with the same tid.

> Also, I'm a wee bit confused, why not swap out the kernel stack with a
> io-wq worker and have the original thread return to userspace. This
> seems like a better defined situation. In so far as anything here is
> well defined.
> 
> Swapping the kernel state seems like a simpler endeavour than swapping
> all that is or might be user visible.

I think we'd just be trading one set of problems for another. In terms
of the kernel side, we have the following set of issues around swapping
the kernel stack instead:

- 'current' itself, this is used throughout the waiting helpers. Things
  like wait_queue_entries that use private = current.

- Any kind of sleeping lock that also stashes away the value of
  'current;.

- signal interrupible sleeps via TASK_KILLABLE

- Just like the userspace side, we have a bunch of kernel side state as
  well in the task_struct. Not a complete list, but:
	- plug, for any kind of in-progress IO submission state
	- journal_info
	- reclaim_state
	- io_uring
	- PF_MEMALLOC and friends
	- task_work
	- lockdep
	- preempt/irq stuff in thread_info

- Kernel stack itself in ways it's tied to the task_struct
	- stack canary
	- KASAN
	- vma tracking
	- thread.sp
	- pt_regs

Probably not a complete list, but it's a start... I do agree that the
kernel stack swap seems like the more immediate idea, but I think it's a
harder problem.

>> Folks that have been around a while may remember earlier
>> attempts at this about 20 years ago.
> 
> That was the whole threadlet thing, right? Very hazy memories of that.
> I'll have to go read that back, I'm sure I still have that in my inbox
> *somewhere*.

Right, and it's actually pretty related to what we're discussing above.
The first one was Zach's fibrils which attempted the kernel stack swap,
and then we had threadlets from Ingo which went the route that this
patch set is also taking and dealt with userspace state swaps instead.

> Surely we abandoned that approach for a reason. ISTR there being some
> significant ick there, much like the thing you're proposing now.

Well like you that state has long since been swapped out on my end, my
recollection was disagreement on the API, not the swapping itself. But
also extremely hazy there...

> Not sure yet on how things compare. Will have to dig through that stuff
> again.

Let me know if you find something interesting there.

-- 
Jens Axboe