Re: [PATCH v6 3/3] iomap: add simple dio path for small direct I/O
From: changfengnan
Date: Tue Jul 07 2026 - 03:09:23 EST
> From: "Joanne Koong"<joannelkoong@xxxxxxxxx>
> Date: Tue, Jul 7, 2026, 03:27
> Subject: Re: [PATCH v6 3/3] iomap: add simple dio path for small direct I/O
> To: "changfengnan"<changfengnan@xxxxxxxxxxxxx>
> Cc: "Christoph Hellwig"<hch@xxxxxxxxxxxxx>, "brauner"<brauner@xxxxxxxxxx>, "djwong"<djwong@xxxxxxxxxx>, "ojaswin"<ojaswin@xxxxxxxxxxxxx>, "dgc"<dgc@xxxxxxxxxx>, "linux-xfs"<linux-xfs@xxxxxxxxxxxxxxx>, "linux-fsdevel"<linux-fsdevel@xxxxxxxxxxxxxxx>, "linux-ext4"<linux-ext4@xxxxxxxxxxxxxxx>, "linux-kernel"<linux-kernel@xxxxxxxxxxxxxxx>, <lidiangang@xxxxxxxxxxxxx>, "pankaj.raghav"<pankaj.raghav@xxxxxxxxx>, "Brian Foster"<bfoster@xxxxxxxxxx>
> On Mon, Jul 6, 2026 at 4:10 AM changfengnan <changfengnan@xxxxxxxxxxxxx> wrote:
> >
> > > From: "Christoph Hellwig"<hch@xxxxxxxxxxxxx>
> > > Date: Fri, Jul 3, 2026, 20:40
> > > Subject: Re: [PATCH v6 3/3] iomap: add simple dio path for small direct I/O
> > > To: "changfengnan"<changfengnan@xxxxxxxxxxxxx>
> > > Cc: "Joanne Koong"<joannelkoong@xxxxxxxxx>, "Christoph Hellwig"<hch@xxxxxxxxxxxxx>, "brauner"<brauner@xxxxxxxxxx>, "djwong"<djwong@xxxxxxxxxx>, "ojaswin"<ojaswin@xxxxxxxxxxxxx>, "dgc"<dgc@xxxxxxxxxx>, "linux-xfs"<linux-xfs@xxxxxxxxxxxxxxx>, "linux-fsdevel"<linux-fsdevel@xxxxxxxxxxxxxxx>, "linux-ext4"<linux-ext4@xxxxxxxxxxxxxxx>, "linux-kernel"<linux-kernel@xxxxxxxxxxxxxxx>, <lidiangang@xxxxxxxxxxxxx>, "pankaj.raghav"<pankaj.raghav@xxxxxxxxx>, "Brian Foster"<bfoster@xxxxxxxxxx>
> > > On Fri, Jul 03, 2026 at 05:13:12PM +0800, changfengnan wrote:
> > > > taken, which introduced some overhead.
> > > > I implemented some optimizations based on Christoph’s suggestions and
> > > > made some modifications to `iomap_process`;
> > > > Now, XFS performance is essentially unchanged, while ext4 still shows a
> > > > 1.5% drop; I will continue to investigate the cause.. I’ve attached the patch.
> > > > Actually, I had expected that switching to `iomap_next` would improve
> > > > performance, since it avoids an indirect call.
> > >
> > > I don't think it will for you - the direct I/O readers don't have
> > > iomap_end methods, so you still have the same number of indirect
> > > calls. What could help is to inline the iterator as seen in Joannes'
> > > fist demo series. We could even look into doing that only for simple
> > > dio in a first step, shifting the burden to use the simple method
> > > to the callers (at least for the POC).
> >
> > I'm not sure if this is what you had in mind, but I made some
> > adjustments referencing Joannes' first patch, and using this approach,
> > there was no performance degradation.
>
> Thanks, Fengnan. This is awesome that it got the degradation down to zero.
>
> Would you be able to test how this change performs on your benchmarks?:
XFS basically has no performance degradation, while ext4 experiences
a 1-1.5% performance degradation.
Round 1:
d67 xfs : 2.04M, 2.05M, 2.04M, 2.04M, 2.04M avg=2.042M
d67 ext4: 2.12M, 2.12M, 2.12M, 2.12M, 2.12M avg=2.120M
joanne86 xfs : 2.13M, 2.05M, 2.04M, 2.05M, 2.05M avg=2.064M
joanne86 ext4: 2.11M, 2.11M, 2.11M, 2.11M, 2.11M avg=2.110M
Round 2:
d67 xfs : 2.05M, 2.05M, 2.05M, 2.05M, 2.05M avg=2.050M
d67 ext4: 2.12M, 2.12M, 2.12M, 2.12M, 2.12M avg=2.120M
joanne86 xfs : 2.04M, 2.04M, 2.04M, 2.04M, 2.05M avg=2.042M
joanne86 ext4: 2.09M, 2.08M, 2.09M, 2.10M, 2.09M avg=2.090M
Round 3:
d67 xfs : 2.04M, 2.04M, 2.04M, 2.04M, 2.05M avg=2.042M
d67 ext4: 2.12M, 2.12M, 2.11M, 2.12M, 2.12M avg=2.118M
joanne86 xfs : 2.04M, 2.05M, 2.04M, 2.04M, 2.04M avg=2.042M
joanne86 ext4: 2.09M, 2.09M, 2.08M, 2.09M, 2.09M avg=2.088M
>
> diff --git a/fs/ext4/file.c b/fs/ext4/file.c
> index dbe073e181a7..bf0c18cf1017 100644
> --- a/fs/ext4/file.c
> +++ b/fs/ext4/file.c
> @@ -91,7 +91,8 @@ static ssize_t ext4_dio_read_iter(struct kiocb
> *iocb, struct iov_iter *to)
> return generic_file_read_iter(iocb, to);
> }
>
> - ret = iomap_dio_rw(iocb, to, ext4_iomap_next, NULL, 0, NULL, 0);
> + ret = iomap_dio_rw(iocb, to, ext4_iomap_next, NULL,
> + IOMAP_DIO_NO_IOMAP_END, NULL, 0);
> inode_unlock_shared(inode);
>
> file_accessed(iocb->ki_filp);
> diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
> index bcd3d4223464..bc3b87e57592 100644
> --- a/fs/iomap/direct-io.c
> +++ b/fs/iomap/direct-io.c
> @@ -968,115 +968,38 @@ static void iomap_dio_simple_end_io(struct bio *bio)
> iocb->ki_complete(iocb, iomap_dio_simple_complete(sr));
> }
>
> -static inline bool
> -iomap_dio_simple_supported(struct kiocb *iocb, struct iov_iter *iter,
> - const struct iomap_dio_ops *dops,
> - unsigned int dio_flags, size_t done_before)
> +static inline void
> +iomap_dio_simple_finish(struct iomap_iter *iomi, iomap_next_fn iomap_next,
> + size_t written)
> {
> - struct inode *inode = file_inode(iocb->ki_filp);
> - size_t count = iov_iter_count(iter);
> -
> - if (dops || done_before)
> - return false;
> - if (iov_iter_rw(iter) != READ)
> - return false;
> - if (!count)
> - return false;
> - /*
> - * Simple dio is an optimization for small IO. Filter out large IO
> - * early as it's the most common case to fail for typical direct IO
> - * workloads.
> - */
> - if (count > inode->i_sb->s_blocksize)
> - return false;
> - if (dio_flags & (IOMAP_DIO_FORCE_WAIT | IOMAP_DIO_PARTIAL |
> - IOMAP_DIO_BOUNCE))
> - return false;
> - if (iocb->ki_pos + count > i_size_read(inode))
> - return false;
> - if (IS_ENCRYPTED(inode))
> - return false;
> -
> - return true;
> + iomi->iter_start_pos = iomi->pos;
> + iomi->pos += written;
> + iomi->len -= written;
> + iomap_next(iomi, &iomi->iomap, &iomi->srcmap);
> }
>
> -/*
> - * Fast path for small, block-aligned direct I/Os that map to a single
> - * contiguous on-disk extent.
> - *
> - * iomap_dio_simple_supported() enforces the cheap up-front constraints before
> - * entering this path.
> - *
> - * @dops must be NULL: a non-NULL @dops means the caller wants its
> - * ->end_io / ->submit_io hooks invoked, and in particular wants its bios to be
> - * allocated from the filesystem-private @dops->bio_set (whose
> front_pad sizes a
> - * filesystem-private wrapper around the bio). The fast path instead allocates
> - * from the shared iomap_dio_simple_pool, whose front_pad matches struct
> - * iomap_dio_simple; the two wrappers are not interchangeable, so we must fall
> - * back to __iomap_dio_rw() in that case.
> - *
> - * @done_before must be zero: a non-zero caller-accumulated residual cannot be
> - * carried through a single-bio inline completion.
> - *
> - * @iter must describe a non-empty READ no larger than the inode block size:
> - * writes, zero-length I/O, and larger requests need the generic iomap direct
> - * I/O path.
> - *
> - * @dio_flags must not request IOMAP_DIO_FORCE_WAIT, IOMAP_DIO_PARTIAL, or
> - * IOMAP_DIO_BOUNCE: this path does not support forced waiting, partial direct
> - * I/O, or bouncing. The range must also stay within i_size and encrypted
> - * inodes must use the generic iomap direct I/O path.
> - *
> - * -ENOTBLK is the private sentinel returned by iomap_dio_simple() when it
> - * decides the request does not fit the fast path. In that case we proceed to
> - * the generic __iomap_dio_rw() slow path. Any other errno is a real
> result and
> - * is propagated as-is, in particular -EAGAIN for IOCB_NOWAIT must reach the
> - * caller.
> - */
> -static ssize_t
> -iomap_dio_simple(struct kiocb *iocb, struct iov_iter *iter,
> - iomap_next_fn iomap_next, void *private,
> - unsigned int dio_flags)
> +ssize_t
> +__iomap_dio_simple(struct kiocb *iocb, struct iov_iter *iter,
> + struct iomap_iter *iomi, iomap_next_fn iomap_next,
> + unsigned int dio_flags)
> {
> struct inode *inode = file_inode(iocb->ki_filp);
> size_t count = iov_iter_count(iter);
> bool wait_for_completion = is_sync_kiocb(iocb);
> - struct iomap_iter iomi = {
> - .inode = inode,
> - .pos = iocb->ki_pos,
> - .len = count,
> - .flags = IOMAP_DIRECT,
> - .private = private,
> - };
> struct iomap_dio_simple *sr;
> unsigned int alignment;
> struct bio *bio;
> ssize_t ret;
>
> - if (iocb->ki_flags & IOCB_NOWAIT)
> - iomi.flags |= IOMAP_NOWAIT;
> -
> - ret = kiocb_write_and_wait(iocb, count);
> - if (ret)
> - return ret;
> -
> - inode_dio_begin(inode);
> -
> - ret = iomap_iter(&iomi, iomap_next);
> - if (ret <= 0) {
> - inode_dio_end(inode);
> - return ret ? ret : -EFAULT;
> - }
> -
> - if (iomi.iomap.type != IOMAP_MAPPED ||
> - iomi.iomap.offset + iomi.iomap.length < iomi.pos + count ||
> - (iomi.iomap.flags & IOMAP_F_INTEGRITY)) {
> + if (iomi->iomap.type != IOMAP_MAPPED ||
> + iomi->iomap.offset + iomi->iomap.length < iomi->pos + count ||
> + (iomi->iomap.flags & IOMAP_F_INTEGRITY)) {
> ret = -ENOTBLK;
> goto out_iomap_end;
> }
>
> - alignment = iomap_dio_alignment(inode, iomi.iomap.bdev, dio_flags);
> - if ((iomi.pos | count) & (alignment - 1)) {
> + alignment = iomap_dio_alignment(inode, iomi->iomap.bdev, dio_flags);
> + if ((iomi->pos | count) & (alignment - 1)) {
> ret = -EINVAL;
> goto out_iomap_end;
> }
> @@ -1092,14 +1015,14 @@ iomap_dio_simple(struct kiocb *iocb, struct
> iov_iter *iter,
> if (user_backed_iter(iter))
> dio_flags |= IOMAP_DIO_USER_BACKED;
>
> - bio = bio_alloc_bioset(iomi.iomap.bdev,
> + bio = bio_alloc_bioset(iomi->iomap.bdev,
> bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS),
> REQ_OP_READ, GFP_KERNEL, &iomap_dio_simple_pool);
> sr = container_of(bio, struct iomap_dio_simple, bio);
> sr->iocb = iocb;
> sr->dio_flags = dio_flags;
>
> - bio->bi_iter.bi_sector = iomap_sector(&iomi.iomap, iomi.pos);
> + bio->bi_iter.bi_sector = iomap_sector(&iomi->iomap, iomi->pos);
> bio->bi_ioprio = iocb->ki_ioprio;
>
> ret = bio_iov_iter_get_pages(bio, iter, alignment - 1);
> @@ -1124,8 +1047,8 @@ iomap_dio_simple(struct kiocb *iocb, struct
> iov_iter *iter,
> WRITE_ONCE(iocb->private, bio);
> }
>
> - iomap_iter_advance(&iomi, count);
> - iomap_iter(&iomi, iomap_next);
> + if (!(dio_flags & IOMAP_DIO_NO_IOMAP_END))
> + iomap_dio_simple_finish(iomi, iomap_next, count);
>
> if (!wait_for_completion) {
> bio->bi_end_io = iomap_dio_simple_end_io;
> @@ -1142,33 +1065,12 @@ iomap_dio_simple(struct kiocb *iocb, struct
> iov_iter *iter,
> out_bio_put:
> bio_put(bio);
> out_iomap_end:
> - iomap_iter(&iomi, iomap_next);
> + if (!(dio_flags & IOMAP_DIO_NO_IOMAP_END))
> + iomap_dio_simple_finish(iomi, iomap_next, 0);
> inode_dio_end(inode);
> return ret;
> }
> -
> -ssize_t
> -iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
> - iomap_next_fn iomap_next, const struct iomap_dio_ops *dops,
> - unsigned int dio_flags, void *private, size_t done_before)
> -{
> - struct iomap_dio *dio;
> - ssize_t ret;
> -
> - if (iomap_dio_simple_supported(iocb, iter, dops, dio_flags,
> - done_before)) {
> - ret = iomap_dio_simple(iocb, iter, iomap_next,
> private, dio_flags);
> - if (ret != -ENOTBLK)
> - return ret;
> - }
> -
> - dio = __iomap_dio_rw(iocb, iter, iomap_next, dops, dio_flags, private,
> - done_before);
> - if (IS_ERR_OR_NULL(dio))
> - return PTR_ERR_OR_ZERO(dio);
> - return iomap_dio_complete(dio);
> -}
> -EXPORT_SYMBOL_GPL(iomap_dio_rw);
> +EXPORT_SYMBOL_GPL(__iomap_dio_simple);
>
> static int __init iomap_dio_init(void)
> {
> diff --git a/fs/iomap/iter.c b/fs/iomap/iter.c
> index 984045af310a..1128f5b526ef 100644
> --- a/fs/iomap/iter.c
> +++ b/fs/iomap/iter.c
> @@ -89,9 +89,6 @@ int iomap_iter_continue(const struct iomap_iter
> *iter, struct iomap *iomap,
> bool stale = iomap->flags & IOMAP_F_STALE;
> ssize_t advanced = iter->pos - iter->iter_start_pos;
>
> - if (!iomap->length)
> - return 1;
> -
> /*
> * Use iter->len to determine whether to continue onto the next mapping.
> * Explicitly terminate on error status or if the current iter has not
> diff --git a/fs/iomap/trace.h b/fs/iomap/trace.h
> index e1ea8392cf47..e43ca419b13a 100644
> --- a/fs/iomap/trace.h
> +++ b/fs/iomap/trace.h
> @@ -127,7 +127,8 @@ DEFINE_RANGE_EVENT(iomap_zero_iter);
> {IOMAP_DIO_FORCE_WAIT, "DIO_FORCE_WAIT" }, \
> {IOMAP_DIO_OVERWRITE_ONLY, "DIO_OVERWRITE_ONLY" }, \
> {IOMAP_DIO_PARTIAL, "DIO_PARTIAL" }, \
> - {IOMAP_DIO_FSBLOCK_ALIGNED, "DIO_FSBLOCK_ALIGNED" }
> + {IOMAP_DIO_FSBLOCK_ALIGNED, "DIO_FSBLOCK_ALIGNED" }, \
> + {IOMAP_DIO_NO_IOMAP_END, "DIO_NO_IOMAP_END" }
>
> DECLARE_EVENT_CLASS(iomap_class,
> TP_PROTO(struct inode *inode, struct iomap *iomap),
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index a987ffbf3c02..6a551efff66f 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -251,7 +251,7 @@ xfs_file_dio_read(
> struct iov_iter *to)
> {
> struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp));
> - unsigned int dio_flags = 0;
> + unsigned int dio_flags = IOMAP_DIO_NO_IOMAP_END;
> const struct iomap_dio_ops *dio_ops = NULL;
> ssize_t ret;
>
> diff --git a/include/linux/iomap.h b/include/linux/iomap.h
> index 3b41f123a92d..6213a3b1a0b9 100644
> --- a/include/linux/iomap.h
> +++ b/include/linux/iomap.h
> @@ -10,6 +10,7 @@
> #include <linux/mm_types.h>
> #include <linux/blkdev.h>
> #include <linux/folio_batch.h>
> +#include <linux/pagemap.h>
>
> struct address_space;
> struct fiemap_extent_info;
> @@ -605,15 +606,136 @@ struct iomap_dio_ops {
> */
> #define IOMAP_DIO_BOUNCE (1 << 4)
>
> -ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
> - iomap_next_fn iomap_next, const struct iomap_dio_ops *dops,
> - unsigned int dio_flags, void *private, size_t done_before);
> +/* optimization hint for the simple fast path */
> +#define IOMAP_DIO_NO_IOMAP_END (1 << 5)
> +
> struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
> iomap_next_fn iomap_next, const struct iomap_dio_ops *dops,
> unsigned int dio_flags, void *private, size_t done_before);
> ssize_t iomap_dio_complete(struct iomap_dio *dio);
> void iomap_dio_bio_end_io(struct bio *bio);
>
> +static inline bool
> +iomap_dio_simple_supported(struct kiocb *iocb, struct iov_iter *iter,
> + const struct iomap_dio_ops *dops,
> + unsigned int dio_flags, size_t done_before)
> +{
> + struct inode *inode = file_inode(iocb->ki_filp);
> + size_t count = iov_iter_count(iter);
> +
> + if (dops || done_before)
> + return false;
> + if (iov_iter_rw(iter) != READ)
> + return false;
> + if (!count)
> + return false;
> + if (count > inode->i_sb->s_blocksize)
> + return false;
> + if (dio_flags & (IOMAP_DIO_FORCE_WAIT | IOMAP_DIO_PARTIAL |
> + IOMAP_DIO_BOUNCE))
> + return false;
> + if (iocb->ki_pos + count > i_size_read(inode))
> + return false;
> + if (IS_ENCRYPTED(inode))
> + return false;
> + return true;
> +}
> +
> +ssize_t __iomap_dio_simple(struct kiocb *iocb, struct iov_iter *iter,
> + struct iomap_iter *iomi, iomap_next_fn iomap_next,
> + unsigned int dio_flags);
> +
> +/*
> + * Fast path for small, block-aligned direct I/Os that map to a single
> + * contiguous on-disk extent.
> + *
> + * iomap_dio_simple_supported() enforces the cheap up-front constraints before
> + * entering this path.
> + *
> + * @dops must be NULL: a non-NULL @dops means the caller wants its
> + * ->end_io / ->submit_io hooks invoked, and in particular wants its bios to be
> + * allocated from the filesystem-private @dops->bio_set (whose
> front_pad sizes a
> + * filesystem-private wrapper around the bio). The fast path instead allocates
> + * from the shared iomap_dio_simple_pool, whose front_pad matches struct
> + * iomap_dio_simple; the two wrappers are not interchangeable, so we must fall
> + * back to __iomap_dio_rw() in that case.
> + *
> + * @done_before must be zero: a non-zero caller-accumulated residual cannot be
> + * carried through a single-bio inline completion.
> + *
> + * @iter must describe a non-empty READ no larger than the inode block size:
> + * writes, zero-length I/O, and larger requests need the generic iomap direct
> + * I/O path.
> + *
> + * @dio_flags must not request IOMAP_DIO_FORCE_WAIT, IOMAP_DIO_PARTIAL, or
> + * IOMAP_DIO_BOUNCE: this path does not support forced waiting, partial direct
> + * I/O, or bouncing. The range must also stay within i_size and encrypted
> + * inodes must use the generic iomap direct I/O path. IOMAP_DIO_NO_IOMAP_END is
> + * a fast-path optimization the caller can set if there is no work
> that needs to
> + * be done after a mapping.
> + *
> + * -ENOTBLK is the private sentinel returned by iomap_dio_simple() when it
> + * decides the request does not fit the fast path. In that case we proceed to
> + * the generic __iomap_dio_rw() slow path. Any other errno is a real
> result and
> + * is propagated as-is, in particular -EAGAIN for IOCB_NOWAIT must reach the
> + * caller.
> + */
> +static __always_inline ssize_t
> +iomap_dio_simple(struct kiocb *iocb, struct iov_iter *iter,
> + iomap_next_fn iomap_next, void *private, unsigned int dio_flags)
> +{
> + struct inode *inode = file_inode(iocb->ki_filp);
> + size_t count = iov_iter_count(iter);
> + struct iomap_iter iomi = {
> + .inode = inode,
> + .pos = iocb->ki_pos,
> + .len = count,
> + .flags = IOMAP_DIRECT,
> + .private = private,
> + };
> + ssize_t ret;
> +
> + if (iocb->ki_flags & IOCB_NOWAIT)
> + iomi.flags |= IOMAP_NOWAIT;
> +
> + ret = kiocb_write_and_wait(iocb, count);
> + if (ret)
> + return ret;
> +
> + inode_dio_begin(inode);
> +
> + ret = iomap_next(&iomi, &iomi.iomap, &iomi.srcmap);
> + if (ret <= 0) {
> + inode_dio_end(inode);
> + /* a zero return means no mapping, which should never happen */
> + return ret ? ret : -EFAULT;
> + }
> +
> + return __iomap_dio_simple(iocb, iter, &iomi, iomap_next, dio_flags);
> +}
> +
> +static __always_inline ssize_t
> +iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, iomap_next_fn
> iomap_next,
> + const struct iomap_dio_ops *dops, unsigned int dio_flags,
> + void *private, size_t done_before)
> +{
> + struct iomap_dio *dio;
> + ssize_t ret;
> +
> + if (iomap_dio_simple_supported(iocb, iter, dops, dio_flags,
> done_before)) {
> + ret = iomap_dio_simple(iocb, iter, iomap_next, private,
> + dio_flags);
> + if (ret != -ENOTBLK)
> + return ret;
> + }
> +
> + dio = __iomap_dio_rw(iocb, iter, iomap_next, dops, dio_flags, private,
> + done_before);
> + if (IS_ERR_OR_NULL(dio))
> + return PTR_ERR_OR_ZERO(dio);
> + return iomap_dio_complete(dio);
> +}
> +
> #ifdef CONFIG_SWAP
> struct file;
> struct swap_info_struct;
> @@ -678,21 +800,22 @@ static __always_inline int iomap_process(const
> struct iomap_iter *iter,
> {
> int ret = 0;
>
> - if (iomap->length && end) {
> - ssize_t advanced = iter->pos - iter->iter_start_pos;
> - loff_t len;
> + if (iomap->length) {
> + if (end) {
> + ssize_t advanced = iter->pos - iter->iter_start_pos;
> + loff_t len;
>
> - len = iomap_length_trim(iter, iter->iter_start_pos,
> - iter->len + advanced);
> + len = iomap_length_trim(iter, iter->iter_start_pos,
> + iter->len + advanced);
>
> - ret = end(iter->inode, iter->iter_start_pos, len, advanced,
> - iter->flags, iomap);
> + ret = end(iter->inode, iter->iter_start_pos,
> len, advanced,
> + iter->flags, iomap);
> + }
> + ret = iomap_iter_continue(iter, iomap, srcmap, ret);
> + if (ret <= 0)
> + return ret;
> }
>
> - ret = iomap_iter_continue(iter, iomap, srcmap, ret);
> - if (ret <= 0)
> - return ret;
> -
> ret = begin(iter->inode, iter->pos, iter->len, iter->flags, iomap,
> srcmap);
>
> I put the changes in a git tree [1] in case that's is more convenient
> for you. I think this ends up devirtualizing the iomap_next callback:
>
> before:
> $ objdump -dr fs/xfs/xfs.ko | awk '/<xfs_file_dio_read>:/{f=1}
> f{print} f&&/^$/{exit}' | grep -E 'iomap_next|iomap_begin|thunk'
> a9ac3: R_X86_64_PLT32 __x86_return_thunk-0x4
> a9b4b: R_X86_64_32S xfs_read_iomap_next
> a9b9c: R_X86_64_PLT32 __x86_return_thunk-0x4
>
> after:
> $ objdump -dr fs/xfs/xfs_iomap_next_inlined.ko | awk
> '/<xfs_file_dio_read>:/{f=1} f{print} f&&/^$/{exit}' | grep -E
> 'iomap_next|iomap_begin|thunk'
> aa083: R_X86_64_PLT32 __x86_return_thunk-0x4
> aa1d4: R_X86_64_PLT32 xfs_read_iomap_next-0x4
> aa1ec: R_X86_64_32S xfs_read_iomap_next
> aa232: R_X86_64_32S xfs_read_iomap_next
>
>
> I was unsure how much bloat this adds but I didn't see a noticable
> impact when I ran ./scripts/bloat-o-meter:
>
> $ ./scripts/bloat-o-meter fs/xfs/xfs.ko fs/xfs/xfs_iomap_next_inlined.ko
> add/remove: 0/0 grow/shrink: 11/1 up/down: 589/-4 (585)
> Function old new delta
> xfs_file_dio_read 309 685 +376
> xfs_file_dio_write_aligned 355 402 +47
> xfs_file_dio_write_unaligned 545 591 +46
> xfs_file_dio_write_atomic 457 502 +45
> xfs_dax_write_iomap_next 267 291 +24
> xfs_atomic_write_cow_iomap_next 1151 1175 +24
> xfs_zoned_direct_write_iomap_next 217 224 +7
> xfs_direct_write_iomap_next 77 84 +7
> xfs_buffered_write_iomap_next 373 379 +6
> xfs_xattr_iomap_next 360 365 +5
> xfs_seek_iomap_next 665 667 +2
> xfs_read_iomap_next 681 677 -4
> Total: Before=1977941, After=1978526, chg +0.03%
>
> where the baseline xfs.ko was built at the commit in [2] and the
> xfs_iomap_next_inlined.ko was built at the commit in [3]. If these
> changes are acceptable to you, then I'll put this into v3 of the
> iomap_next series.
>
> Thanks,
> Joanne
>
> [1] https://github.com/joannekoong/linux/commits/iomap_simple_dio_iomap_next/
> [2] https://github.com/joannekoong/linux/commit/1e82c8988ee56e9e7202d5ee09c7f7ea7ac7a832
> [3] https://github.com/joannekoong/linux/commit/86ecc9de53bb1b8b1287950fbb745a5a8e49f736
>