Re: [PATCH] ublk: skip blk_mq_tag_to_rq() bounds check

[Caleb Sander Mateos]

On Thu, Apr 10, 2025 at 6:13 AM Jens Axboe <axboe@xxxxxxxxx> wrote:
>
> On 4/10/25 3:24 AM, Christoph Hellwig wrote:
> > On Tue, Apr 08, 2025 at 08:49:54PM -0600, Caleb Sander Mateos wrote:
> >> The ublk driver calls blk_mq_tag_to_rq() in several places.
> >> blk_mq_tag_to_rq() tolerates an invalid tag for the tagset, checking it
> >> against the number of tags and returning NULL if it is out of bounds.
> >> But all the calls from the ublk driver have already verified the tag
> >> against the ublk queue's queue depth. In ublk_commit_completion(),
> >> ublk_handle_need_get_data(), and case UBLK_IO_COMMIT_AND_FETCH_REQ, the
> >> tag has already been checked in __ublk_ch_uring_cmd(). In
> >> ublk_abort_queue(), the loop bounds the tag by the queue depth. In
> >> __ublk_check_and_get_req(), the tag has already been checked in
> >> __ublk_ch_uring_cmd(), in the case of ublk_register_io_buf(), or in
> >> ublk_check_and_get_req().
> >>
> >> So just index the tagset's rqs array directly in the ublk driver.
> >> Convert the tags to unsigned, as blk_mq_tag_to_rq() does.
> >
> > Poking directly into block layer internals feels like a really bad
> > idea.  If this is important enough we'll need a non-checking helper
> > in the core code, but as with all these kinds of micro-optimizations
> > it better have a really good justification.
>
> FWIW, I agree, and I also have a hard time imagining this making much of
> a measurable difference. Caleb, was this based "well this seems
> pointless" or was it something you noticed in profiling/testing?

That's true, the nr_tags check doesn't show up super prominently in a
CPU profile. The atomic reference counting in
__ublk_check_and_get_req() or ublk_commit_completion() is
significantly more expensive. Still, it seems like unnecessary work.
nr_tags is in a different cache line from rqs, so there is the
potential for a cache miss. And the prefetch() is another unnecessary
cache miss in the cases where ublk doesn't access any of struct
request's fields.
I am happy to add a "blk_mq_tag_to_rq_unchecked()" helper to avoid
accessing the blk-mq internals.

Best,
Caleb