Re: [PATCH v2 12/12] Documentation: iio: Document high-speed DMABUF based API
From: Paul Cercueil
Date: Tue Mar 29 2022 - 05:47:49 EST
Hi Daniel,
Le mar., mars 29 2022 at 10:54:43 +0200, Daniel Vetter
<daniel@xxxxxxxx> a écrit :
On Mon, Feb 07, 2022 at 01:01:40PM +0000, Paul Cercueil wrote:
Document the new DMABUF based API.
v2: - Explicitly state that the new interface is optional and is
not implemented by all drivers.
- The IOCTLs can now only be called on the buffer FD returned by
IIO_BUFFER_GET_FD_IOCTL.
- Move the page up a bit in the index since it is core stuff
and not
driver-specific.
Signed-off-by: Paul Cercueil <paul@xxxxxxxxxxxxxxx>
---
Documentation/driver-api/dma-buf.rst | 2 +
Documentation/iio/dmabuf_api.rst | 94
++++++++++++++++++++++++++++
Documentation/iio/index.rst | 2 +
3 files changed, 98 insertions(+)
create mode 100644 Documentation/iio/dmabuf_api.rst
diff --git a/Documentation/driver-api/dma-buf.rst
b/Documentation/driver-api/dma-buf.rst
index 2cd7db82d9fe..d3c9b58d2706 100644
--- a/Documentation/driver-api/dma-buf.rst
+++ b/Documentation/driver-api/dma-buf.rst
@@ -1,3 +1,5 @@
+.. _dma-buf:
+
Buffer Sharing and Synchronization
==================================
diff --git a/Documentation/iio/dmabuf_api.rst
b/Documentation/iio/dmabuf_api.rst
new file mode 100644
index 000000000000..43bb2c1b9fdc
--- /dev/null
+++ b/Documentation/iio/dmabuf_api.rst
@@ -0,0 +1,94 @@
+===================================
+High-speed DMABUF interface for IIO
+===================================
+
+1. Overview
+===========
+
+The Industrial I/O subsystem supports access to buffers through a
file-based
+interface, with read() and write() access calls through the IIO
device's dev
+node.
+
+It additionally supports a DMABUF based interface, where the
userspace
+application can allocate and append DMABUF objects to the buffer's
queue.
+This interface is however optional and is not available in all
drivers.
+
+The advantage of this DMABUF based interface vs. the read()
+interface, is that it avoids an extra copy of the data between the
+kernel and userspace. This is particularly useful for high-speed
+devices which produce several megabytes or even gigabytes of data
per
+second.
+
+The data in this DMABUF interface is managed at the granularity of
+DMABUF objects. Reducing the granularity from byte level to block
level
+is done to reduce the userspace-kernelspace synchronization
overhead
+since performing syscalls for each byte at a few Mbps is just not
+feasible.
+
+This of course leads to a slightly increased latency. For this
reason an
+application can choose the size of the DMABUFs as well as how many
it
+allocates. E.g. two DMABUFs would be a traditional double buffering
+scheme. But using a higher number might be necessary to avoid
+underflow/overflow situations in the presence of scheduling
latencies.
So this reads a lot like reinventing io-uring with pre-registered
O_DIRECT
memory ranges. Except it's using dma-buf and hand-rolling a lot of
pieces
instead of io-uring and O_DIRECT.
I don't see how io_uring would help us. It's an async I/O framework,
does it allow us to access a kernel buffer without copying the data?
Does it allow us to zero-copy the data to a network interface?
At least if the entire justification for dma-buf support is zero-copy
support between the driver and userspace it's _really_ not the right
tool
for the job. dma-buf is for zero-copy between devices, with cpu access
from userpace (or kernel fwiw) being very much the exception (and
often
flat-out not supported at all).
We want both. Using dma-bufs for the driver/userspace interface is a
convenience as we then have a unique API instead of two distinct ones.
Why should CPU access from userspace be the exception? It works fine
for IIO dma-bufs. You keep warning about this being a terrible design,
but I simply don't see it.
Cheers,
-Paul
+
+2. User API
+===========
+
+``IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *)``
+----------------------------------------------------------------
+
+Each call will allocate a new DMABUF object. The return value (if
not
+a negative errno value as error) will be the file descriptor of
the new
+DMABUF.
+
+``IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *)``
+--------------------------------------------------------
+
+Place the DMABUF object into the queue pending for hardware
process.
+
+These two IOCTLs have to be performed on the IIO buffer's file
+descriptor, obtained using the `IIO_BUFFER_GET_FD_IOCTL` ioctl.
+
+3. Usage
+========
+
+To access the data stored in a block by userspace the block must be
+mapped to the process's memory. This is done by calling mmap() on
the
+DMABUF's file descriptor.
+
+Before accessing the data through the map, you must use the
+DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the
+DMA_BUF_SYNC_START flag, to make sure that the data is available.
+This call may block until the hardware is done with this block.
Once
+you are done reading or writing the data, you must use this ioctl
again
+with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the
+kernel's queue.
+
+If you need to know when the hardware is done with a DMABUF, you
can
+poll its file descriptor for the EPOLLOUT event.
+
+Finally, to destroy a DMABUF object, simply call close() on its
file
+descriptor.
+
+For more information about manipulating DMABUF objects, see:
:ref:`dma-buf`.
+
+A typical workflow for the new interface is:
+
+ for block in blocks:
+ DMABUF_ALLOC block
+ mmap block
+
+ enable buffer
+
+ while !done
+ for block in blocks:
+ DMABUF_ENQUEUE block
+
+ DMABUF_SYNC_START block
+ process data
+ DMABUF_SYNC_END block
+
+ disable buffer
+
+ for block in blocks:
+ close block
diff --git a/Documentation/iio/index.rst
b/Documentation/iio/index.rst
index 58b7a4ebac51..669deb67ddee 100644
--- a/Documentation/iio/index.rst
+++ b/Documentation/iio/index.rst
@@ -9,4 +9,6 @@ Industrial I/O
iio_configfs
+ dmabuf_api
+
ep93xx_adc
--
2.34.1
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch