Re: [RFC PATCH 00/15] io_uring: thread identity handoff for blocking inline issue

[Jens Axboe]

On 9/18/26 10:33 PM, Eric W. Biederman wrote:
> Jens Axboe <axboe@xxxxxxxxx> writes:
> 
>> Hi,
>>
>> io_uring issues requests inline with IO_URING_F_NONBLOCK and punts to
>> io-wq when that isn't possible. For a range of opcodes it isn't possible
>> at all, as there's no nonblocking path in the kernel for them: fsync,
>> statx, openat, the *at family, xattr, fadvise, splice, etc. Those are
>> punted unconditionally, and the punt costs a thread wakeup, a context
>> switch and a task_work completion round trip per request. io_uring HAS
>> to be cautious to prevent accidental blocking in the kernel, even if the
>> operations predominantly never block. Sad story. Examples of that are
>> things like an fdatasync that doesn't block, statx that hits dcache,
>> openat for O_TMPFILE, etc. All of those would've completed inline just
>> fine, but io_uring just cannot rely on that.
>>
>> This series issues those requests inline in blocking mode instead, and
>> only pays for the offload if the request actually blocks. But by the
>> time it blocks, the submitter is deep in the kernel with the request on
>> its stack, so the work can't be moved to another thread. What we can
>> move is the identity. If the submitting task blocks, an idle io-wq
>> worker takes over its user visible identity (tid, signal state,
>> credentials, scheduling attributes, cgroup, user register state),
>> finishes the io_uring_enter() call and returns to userspace as the
>> submitter. The original task finishes the request as an
>> io-wq worker and joins the pool. Userspace is none the wiser, hopefully,
>> the same tid came back from the syscall, it's just on a different
>> task_struct. Folks that have been around a while may remember earlier
>> attempts at this about 20 years ago.
> 
> I don't see anything immediately wrong, but I suspect I am just
> not looking hard enough.
> 
> In my time working with the kernel I have never seen anyone actually get
> this kind of thing correct.
> 
> The handoff that we do during exec has a bug with posix timers that
> I think is 23 years old that we just caught, and still hasn't been
> merged to Linus.
> 
> There was the old daemonize call that got it wrong so often I added
> kthreadd.

I agree entirely with you, which is why this is (deeply) and RFC and I
mostly pulled it to (some notion of) completion so I could run some
testing and see how it performs.

> Maybe you want something like the old solaris doors, or vfork.
> Perform a synchronous task switch to this other thread, and call this
> function in the other thread.  Then block waiting on the other thread
> until the other thread blocks, or the function you called finishes.
> 
> Is there a reason you didn't try and do it that way?
> Just a synchronous switch to and from a thread in your thread pool?
> 
> You aren't changing the mm so I really doubt changing the stack pointer
> and a registers will be that expensive.

And replies like this are also why I wanted to get it out, because I
think it's a problem worth solving, and it's the best way to solicit
ideas. I think there's some potential in your suggestion, let me try and
dig at it a little bit and experiment...

I'll be back with more details.

-- 
Jens Axboe