Re: [PATCH v2] ceph: acquire caps for read_folio without an rw context

[Xiubo Li]

LGTM.

Reviewed-by: Xiubo Li <xiubo.li@xxxxxxxxx>

On Fri, 4 Sept 2026 at 03:51, Max Kellermann <max.kellermann@xxxxxxxxx> wrote:
>
> This fixes a data corruption problem that leaked permanently into
> fscache.  File-backed erofs reads metadata using
> read_mapping_folio(), and the Ceph implementation of this call forgets
> to acquire Ceph caps.  Therefore, mounting an erofs image file from a
> Ceph mount that was just written (but not yet committed to the OSD)
> would fail because the erofs code saw only zero-filled pages.  These
> zero-filled pages were then copied to the fscache, making this data
> corruption permanent (on this host).
>
> Usually, Ceph checks/acquires caps at its own entry points and not in
> the `address_space_operations`: ceph_read_iter() and
> ceph_filemap_fault() acquire Fr/Fc before calling filemap_read() or
> filemap_fault(), and ceph_write_iter() holds Fw/Fb around write_begin.
>
> Only readahead, which the VM can invoke without passing through a Ceph
> entry point, checks caps itself.  That check was added by
> commit 2b1ac852eb67 ("ceph: try getting buffer capability for
> readahead/fadvise") to the readpages path, converted to use the rw
> context list by commit 5d988308283e ("ceph: track read contexts in
> ceph_file_info"), and moved into ceph_init_request() for
> NETFS_READAHEAD by commit a5c9dc445139 ("ceph: Make
> ceph_init_request() check caps on readahead").
>
> The single-folio read path never had such a check, neither in the old
> ceph_readpage() nor in netfs_read_folio() via ceph_init_request().  It
> assumes that the `read_folio` method is only ever reached from
> filemap_read() or filemap_fault(), both of which Ceph wraps.
>
> However, since Linux 6.12, erofs file-backed mounts
> (commit ce63cb62d794 ("erofs: support unencoded inodes for fileio"))
> read all metadata (including the superblock) by calling
> read_mapping_folio() directly on the backing file's mapping.  On Ceph,
> this issues an OSD read without holding any caps.
>
> The result is silent data corruption.  Ceph clients do not write back
> dirty pages on close(); a writer keeps Fb and its dirty data until the
> MDS revokes the cap.  When another client opens the file, the MDS
> initiates that revoke and replies to the open immediately.  A read()
> would now block in ceph_get_caps() until the writer has flushed and
> acked the cap-revoke, but the erofs superblock read goes to the OSD
> without acquiring caps and thus races with the writeback.  If the
> object does not exist yet, the OSD returns -ENOENT, which
> finish_netfs_read() treats as "success, no data", and netfs zero-fills
> the folio.  The folio is marked uptodate and, because it counts as
> downloaded from the server, is also copied into fscache.  This never
> recovers because Ceph invalidates the page cache and fscache only when
> Fc is revoked, but gaining Fc later will not invalidate it.
>
> Add a Ceph read_folio wrapper which acquires caps synchronously when no
> rw context is present.  Cap acquisition can process a pending truncate,
> so drop the folio lock before acquiring caps.  After relocking, verify
> that the folio still belongs to the mapping and has not already become
> uptodate before passing it to netfs_read_folio().  Leave reads with an
> existing rw context and the legacy inline-data path unchanged.
>
> Fixes: ce63cb62d794 ("erofs: support unencoded inodes for fileio")
> Cc: stable@xxxxxxxxxxxxxxx
> Signed-off-by: Max Kellermann <max.kellermann@xxxxxxxxx>
> ---
> Note:
> - commit e587a984d332 ("erofs: use dedicated meta inodes for
>   file-backed mounts") eliminates this trigger, so Linux 7.3-rc1 and
>   later are not affected.  The fix remains relevant to older stable
>   kernels and to future direct read_mapping_folio() callers.
> v1 -> v2:
> - Move cap acquisition from ceph_init_request() into a read_folio
>   wrapper, leaving the readahead path unchanged.
> - Drop the folio lock around blocking cap acquisition, then relock and
>   revalidate the folio to avoid a truncate deadlock.
> - Preserve the legacy inline-data path.
> - Handle a NULL file argument without acquiring caps a second time.
> - Require Fc before populating the page cache; CEPH_CAP_FILE_LAZYIO
>   cannot be added because the want mask is conjunctive
>
> Signed-off-by: Max Kellermann <max.kellermann@xxxxxxxxx>
> ---
>  fs/ceph/addr.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 57 insertions(+), 1 deletion(-)
>
> diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
> index 657c2cb0f881..78b805ccc5d4 100644
> --- a/fs/ceph/addr.c
> +++ b/fs/ceph/addr.c
> @@ -1959,8 +1959,64 @@ static int ceph_write_end(const struct kiocb *iocb,
>         return copied;
>  }
>
> +static int ceph_read_folio(struct file *file, struct folio *folio)
> +{
> +       struct inode *inode = folio_inode(folio);
> +       struct ceph_inode_info *ci = ceph_inode(inode);
> +       struct ceph_file_info *fi = file ? file->private_data : NULL;
> +       int got = 0;
> +       int ret;
> +
> +       /*
> +        * Existing Ceph read paths acquire caps before entering the page
> +        * cache.  Keep the legacy inline-data path unchanged.
> +        */
> +       if (ceph_has_inline_data(ci) ||
> +           (fi && ceph_find_rw_context(fi)))
> +               return netfs_read_folio(file, folio);
> +
> +       /*
> +        * Cap acquisition can process a pending truncate, which may lock and
> +        * remove this folio.  It must therefore happen without the folio lock.
> +        */
> +       folio_unlock(folio);
> +       ret = __ceph_get_caps(inode, fi, CEPH_CAP_FILE_RD,
> +                             CEPH_CAP_FILE_CACHE, -1, &got);
> +       if (ret < 0)
> +               return ret;
> +
> +       if (!(got & CEPH_CAP_FILE_CACHE)) {
> +               ret = -EACCES;
> +               goto out;
> +       }
> +
> +       folio_lock(folio);
> +
> +       /*
> +        * After re-locking, check if the folio still belongs to the
> +        * mapping...
> +        */
> +       if (folio->mapping != inode->i_mapping) {
> +               folio_unlock(folio);
> +               ret = AOP_TRUNCATED_PAGE;
> +               goto out;
> +       }
> +
> +       /* .. or has been filled already meanwhile */
> +       if (folio_test_uptodate(folio)) {
> +               folio_unlock(folio);
> +               ret = 0;
> +               goto out;
> +       }
> +
> +       ret = netfs_read_folio(file, folio);
> +out:
> +       ceph_put_cap_refs(ci, got);
> +       return ret;
> +}
> +
>  const struct address_space_operations ceph_aops = {
> -       .read_folio = netfs_read_folio,
> +       .read_folio = ceph_read_folio,
>         .readahead = netfs_readahead,
>         .writepages = ceph_writepages_start,
>         .write_begin = ceph_write_begin,
> --
> 2.47.3
>