[PATCH v4 0/5] nfs: modernize direct I/O path
From: Pranjal Shrivastava
Date: Mon Jul 20 2026 - 11:12:45 EST
Modernize the NFS Direct I/O path as a preparatory step to enable PCI
Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial
RFC [1], the modernization and architectural changes are split into
different series. Additionally, based on the discussion in the v2 [2]
of this series, the migration of NFS Direct I/O to folios would be
handled in a separate follow-up series.
Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2()
API which does not support the pinning requirements for P2P memory.
The implementation moves NFS to the modern iov_iter_extract_pages() API.
Design
======
1. Pin-Awareness
Standard NFS requests use get_page() and put_page() for memory
management. However, memory extracted via iov_iter_extract_pages()
requires explicit pinning.
Introduce a PG_PINNED flag and a wb_nr_pinned count to struct nfs_page.
This allows the request lifecycle to track ownership of physical pins
and ensure that unpinning is performed only when the I/O is complete.
2. API Migration
Migrate the Direct I/O path to the modern iov_iter_extract_pages()
API. This aligns NFS with the modern extraction model and serves as
the foundation for passing ITER_ALLOW_P2PDMA in a follow-up series.
Upcoming Work / Roadmap
================================
As decided in the RFC [1] & v2 [2], there will be separate series for
P2PDMA Enablement and Migrating NFS Direct I/O to use folios.
This series lays the necessary groundwork for the upcoming work.
Following this, two additional series are planned:
1. Migrating NFS Direct I/O to Folios
A series that introduces and exports helper from iov_iter.c to allow
the nfs_direct_extract_pages() helper introduced here to aggregate
multiple pages into a single large folio-based request, aiming to reduce
the RPC overhead for hugepage I/O.
2. P2PDMA Enablement for NFS
Enabling ITER_ALLOW_P2PDMA for the Direct I/O path and introducing transport-level
negotiation (discovery of P2P-capable RDMA/NVMe devices). This will build
upon the PG_PINNED infrastructure introduced by this series.
Testing
=======
This series has been tested with xfstests [3] on RDMA & TCP transports
by running the quick test suite for each transport vs. version combo:
./check -g quick -s rdma3 -s rdma40 -s rdma41 -s rdma42 -s tcp3 -s tcp40 -s tcp41 -s tcp42
The tests were run before & after applying the series. No regressions were observed.
The following summary was tabulated via a custom script [4] (on github)
to depict that the tests failing in v1 [5] are now passing.
python3 display.py results/*/check.log
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
| testcase | rdma-sys-3 | rdma-sys-4.0 | rdma-sys-4.1 | rdma-sys-4.2 | tcp-sys-3 | tcp-sys-4.0 | tcp-sys-4.1 | tcp-sys-4.2 |
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
| generic/091 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/130 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/139 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/143 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/154 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/155 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/183 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/188 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/190 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/196 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/198 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/203 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/214 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/240 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/263 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/287 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/290 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/292 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/330 | skipped | skipped | skipped | pass | skipped | skipped | skipped | pass |
| generic/444 | skipped | skipped | skipped | skipped | skipped | skipped | skipped | skipped |
| generic/450 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/451 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/586 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/647 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/708 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/729 | pass | pass | pass | pass | pass | pass | pass | pass |
| generic/760 | pass | pass | pass | pass | pass | pass | pass | pass |
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
Thanks,
Praan
[1] https://lore.kernel.org/all/20260401194501.2269200-1-praan@xxxxxxxxxx/
[2] https://lore.kernel.org/all/ak8-NMsNPOB3zpF-@xxxxxxxxxxxxx/
[3] https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
[4] https://github.com/pran005/tools/blob/main/display.py
[5] https://lore.kernel.org/all/29a0511d-5216-46f2-a7e4-9c04ae9b1890@xxxxxxxxxxxxxxxx/
[v4]
- Dropped superfluous externs in include/linux/nfs_page.h
- Scoped `req` locally inside the loop in nfs_release_request_list()
- Wrapped >80 character lines in Patch 1 to adhere to coding style.
- Fixed typos and capitalization convention in the commit msgs
- Collected Reviewed-by tags from Christoph and Shivaji
- Rebased on fs-next
[v3]
- https://lore.kernel.org/all/20260715143540.3597616-1-praan@xxxxxxxxxx/
- Dropped patches that added folio support for NFS Direct I/O
- Folded requested_bytes accounting in patch 5 due to dropped folio support
- Rebased on fs-next
[v2]
- https://lore.kernel.org/all/20260616134000.2733403-1-praan@xxxxxxxxxx/
- Fix data corruption in nfs_direct_extract_pages() by correctly
calculating intra-page offsets using offset_in_page().
- Fix requested_bytes accounting in direct read/write paths to only
increment after successful RPC scheduling.
- Add missing kernel-doc descriptions for the @pinned parameter in
nfs_page_create_from_page() and nfs_page_create_from_folio().
- Rebase on fs-next/
[v1] https://lore.kernel.org/all/20260603053033.3300318-1-praan@xxxxxxxxxx/
Pranjal Shrivastava (5):
nfs: make nfs_page pin-aware
nfs: track number of pinned pages in nfs_page
nfs: introduce nfs_release_request_list helper
nfs: migrate direct I/O to iov_iter_extract_pages
nfs: introduce nfs_direct_extract_pages helper
fs/nfs/direct.c | 143 ++++++++++++++++++++-------------------
fs/nfs/pagelist.c | 62 ++++++++++++++---
fs/nfs/read.c | 2 +-
fs/nfs/write.c | 2 +-
include/linux/nfs_page.h | 8 ++-
5 files changed, 135 insertions(+), 82 deletions(-)
base-commit: 070f41f072722437b7edda13037a400459630a5e
--
2.55.0.229.g6434b31f56-goog