Re: [PATCH V6 00/17] io_uring/ublk: add generic IORING_OP_FUSED_CMD

[Bernd Schubert]

On 4/20/23 03:38, Ming Lei wrote:
> On Wed, Apr 19, 2023 at 03:42:40PM +0000, Bernd Schubert wrote:
>> I very much see the use case for FUSED_CMD for overlay or simple network
>> sockets. Now in the HPC world one typically uses IB  RDMA and if that
>> fails for some reasons (like connection down), tcp or other interfaces
>> as fallback. And there is sending the right part of the buffer to the
>> right server and erasure coding involved - it gets complex and I don't
>> think there is a way for us without a buffer copy.
> 
> As I mentioned, it(checksum, encrypt, ...) becomes one generic issue if
> the zero copy approach is accepted, meantime the problem itself is well-defined,
> so I don't worry no solution can be figured out.
> 
> Meantime big memory copy does consume both cpu and memory bandwidth a
> lot, and 64k/512k ublk io has shown this big difference wrt. copy vs.
> zero copy.

I don't have any doubt about that, but I believe there is no current way 
to support it in all use cases. As example, let's consider we would like 
to extend nbd with verbs/rdma instead of plain tcp  - verbs/rdma needs 
registered memory and does not take a simple socket fd to send buffers to.


Thanks,
Bernd