Re: FW: [PATCH 0/3] Demand faulting for huge pages

[Andrew Morton]

Adam Litke <agl@xxxxxxxxxx> wrote:
>
>  Honestly, I think there is an even more fundamental issue at hand.  If
>  the goal is transparent and flexible use of huge pages it seems to me
>  that there is two ways to go:
> 
>  1) Continue with hugetlbfs and work to finish implementing all of the
>  operations (that make sense) properly (like read, write, truncate, etc).

hugetlbfs provides the API by which applications may obtain
hugetlb-page-backed memory.  In fact the filesystem didn't even exist in the
initial version of the patch - the first version used specific syscalls to
obtain the hugepage memory.

So.  Given that hugetlbfs is purely there as a means by which applications
can access (and share) hugepage memory, it doesn't make sense to flesh that
filesystem out any further.  IOW: no need for read() and write().

>  2) Recognize that trying to use hugetlbfs files to transparently replace
>  normal memory is ultimately a hack.  Normal memory is not implemented as
>  a file system so using hugetlb pages here will always cause headaches as
>  implemented.  So work towards removing filesystem-like behaviour and
>  treating huge pages more like regular memory.

Early Linus diktat was that we shouldn't attempt to make the core MM aware
of multiple page sizes in the manner which you suggest.  Trying to sneak
this in via "improved integration of hugepage support" would likely create
a mess.

The design approach for hugepage integration was that the MM would continue
to be focussed on a fixed page size and that hugepages would be some
non-intrusive thing off to the side - more like a mmappable device driver
than some core part of the MM system.

This is not all meant to say "don't do it".  But I am saying that you'll
need to review several years worth of discussion on the topic and
understand the downsides and objections, and be prepared for a big project.
One which risks causing Hugh a ton of grief in ongoing core MM
improvements.

Aside: one problem with the kernel's hugepage support is that it doesn't
have a single person who performs the overall maintenance function.  Bill
Irwin was doing this for a while, but now seems to have gone quiet. 

Consequently various people come in and attempt various
this-is-a-change-i-need operations.  Problem is, with no single person
keeping track of who the affected stakeholders are, and what the likely
effects of each change upon the stakeholders will be, things proceed slowly
and various people end up maintaining various out-of-tree things (I think).

I attempt to plug the gaps, but the time interval between flurries of
hugetlb activity are long and I forget who's doing what.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/