Re: [QUESTION] Sharing a `struct page` across multiple `struct address_space` instances

[Matthew Wilcox]

On Fri, Jul 24, 2020 at 05:22:21PM -0700, Vito Caputo wrote:
> Prior to looking at the code, conceptually I was envisioning the pages
> in the reflink source inode's address_space would simply get their
> refcounts bumped as they were added to the dest inode's address_space,
> with some CoW flag set to prevent writes.
> 
> But there seems to be a fundamental assumption that a `struct page`
> would only belong to a single `struct address_space` at a time, as it
> has single `mapping` and `index` members for reverse mapping the page
> to its address_space.
> 
> Am I completely lost here or does it really look like a rather
> invasive modification to support this feature?
> 
> I have vague memories of Dave Chinner mentioning work towards sharing
> pages across address spaces in the interests of getting reflink copies
> more competitive with overlayfs in terms of page cache utilization.

It's invasive.  Dave and I have chatted about this in the past.  I've done
no work towards it (... a little busy right now with THPs in the page
cache ...) but I have a design in mind.

The fundamental idea is to use the DAX support to refer to pages which
actually belong to a separate address space.  DAX entries are effectively
PFN entries.  So there would be a clear distinction between "I looked
up a page which actually belongs to this address space" and "I looked
up a page which is shared with a different address space".  My thinking
has been that if files A and B are reflinked, both A and B would see
DAX entries in their respective page caches.  The page would belong to
a third address space which might be the block device's address space,
or maybe there would be an address space per shared fragment (since
files can share fragments that are at different offsets from each other).

There are a lot of details to get right around this approach.
Importantly, there _shouldn't_ be a refcount from each of file A and
B on the page.  Instead the refcount from files A and B should be on
the fragment.  When the fragment's refcount goes to zero, we know there
are no more references to the fragment and all its pages can be freed.

That means that if we reflink B to C, we don't have to walk every page
in the file and increase its refcount again.

So, are you prepared to do a lot of work, or were you thinking this
would be a quick hack?  Because I'm willing to advise on a big project,
but if you're thinking this will be quick, and don't have time for a
big project, it's probably time to stop here.

---

Something that did occur to me while writing this is that if you just want
read-only duplicates of files to work, you could make inode->i_mapping
point to a different address_space instead of &inode->i_data.  There's
probabyl a quick hack solution there.