On 8/28/14, Maxim Patlasov <mpatlasov@xxxxxxxxxxxxx> wrote:
On 08/21/2014 09:44 AM, Ming Lei wrote:If you enable BLK_MQ_F_WQ_CONTEXT, it isn't strange to see this
On Wed, Aug 20, 2014 at 4:50 AM, Jens Axboe <axboe@xxxxxxxxx> wrote:I'm interested what's the price of handling requests in a separate
Reworked a bit more:One big problem of the commit is that it is basically a serialized
because of single &hctx->run_work, and per-req work_struct has to be
used for concurrent implementation. So looks the approach isn't flexible
enough compared with doing that in driver, or any idea about how to fix
thread at large. I used the following fio script:
filename=/dev/loop0 # or /dev/nullb0
to compare the performance of:
1) /dev/loop0 of 3.17.0-rc1 with Ming's patches applied -- 11K iops
result since blk-mq implements a serialized workqueue.
2) the same as above, but call loop_queue_work() directly fromIn my recent investigation and discussion with Jens, using workqueue
loop_queue_rq() -- 270K iops
3) /dev/nullb0 of 3.17.0-rc1 -- 380K iops
may introduce some regression for cases like loop over null_blk, tmpfs.
And 270K vs. 380K is a bit similar with my result, and it was observed that
context switch is increased by more than 50% with introducing workqueue.
I will post V3 which will use previous kthread, with blk-mq & kernel aio, which
should make full use of blk-mq and kernel aio, and won't introduce regression
for cases like above.
Taking into account so big difference (11K vs. 270K), would it be worthyThe kernel aio submit is very similar with user space's implementation,
to implement pure non-blocking version of aio_kernel_submit() returning
error if blocking needed? Then loop driver (or any other in-kernel user)
except for block plug&unplug usage in user space aio submit path.
If it is blocked in aio_kernel_submit(), you should observe similar thing
with io_submit() too.