Re: [RFC PATCH 00/26] hugetlb: Introduce HugeTLB high-granularity mapping

[Nadav Amit]

> On Jun 27, 2022, at 1:31 PM, James Houghton <jthoughton@xxxxxxxxxx> wrote:
> 
> ⚠ External Email
> 
> On Mon, Jun 27, 2022 at 10:56 AM Dr. David Alan Gilbert
> <dgilbert@xxxxxxxxxx> wrote:
>> 
>> * James Houghton (jthoughton@xxxxxxxxxx) wrote:
>>> On Fri, Jun 24, 2022 at 11:29 AM Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:
>>>> 
>>>> On Fri, Jun 24, 2022 at 05:36:30PM +0000, James Houghton wrote:
>>>>> [1] This used to be called HugeTLB double mapping, a bad and confusing
>>>>> name. "High-granularity mapping" is not a great name either. I am open
>>>>> to better names.
>>>> 
>>>> Oh good, I was grinding my teeth every time I read it ;-)
>>>> 
>>>> How does "Fine granularity" work for you?
>>>> "sub-page mapping" might work too.
>>> 
>>> "Granularity", as I've come to realize, is hard to say, so I think I
>>> prefer sub-page mapping. :) So to recap the suggestions I have so far:
>>> 
>>> 1. Sub-page mapping
>>> 2. Granular mapping
>>> 3. Flexible mapping
>>> 
>>> I'll pick one of these (or maybe some other one that works better) for
>>> the next version of this series.
>> 
>> <shrug> Just a name; SPM might work (although may confuse those
>> architectures which had subprotection for normal pages), and at least
>> we can mispronounce it.
>> 
>> In 14/26 your commit message says:
>> 
>> 1. Faults can be passed to handle_userfault. (Userspace will want to
>> use UFFD_FEATURE_REAL_ADDRESS to get the real address to know which
>> region they should be call UFFDIO_CONTINUE on later.)
>> 
>> can you explain what that new UFFD_FEATURE does?
> 
> +cc Nadav Amit <namit@xxxxxxxxxx> to check me here.
> 
> Sorry, this should be UFFD_FEATURE_EXACT_ADDRESS. It isn't a new
> feature, and it actually isn't needed (I will correct the commit
> message). Why it isn't needed is a little bit complicated, though. Let
> me explain:
> 
> Before UFFD_FEATURE_EXACT_ADDRESS was introduced, the address that
> userfaultfd gave userspace for HugeTLB pages was rounded down to be
> hstate-size-aligned. This would have had to change, because userspace,
> to take advantage of HGM, needs to know which 4K piece to install.
> 
> However, after UFFD_FEATURE_EXACT_ADDRESS was introduced[1], the
> address was rounded down to be PAGE_SIZE-aligned instead, even if the
> flag wasn't used. I think this was an unintended change. If the flag
> is used, then the address isn't rounded at all -- that was the
> intended purpose of this flag. Hope that makes sense.
> 
> The new userfaultfd feature, UFFD_FEATURE_MINOR_HUGETLBFS_HGM, informs
> userspace that high-granularity CONTINUEs are available.
> 
> [1] commit 824ddc601adc ("userfaultfd: provide unmasked address on page-fault")

Indeed this change of behavior (not aligning to huge-pages when flags is
not set) was unintentional. If you want to fix it in a separate patch so
it would be backported, that may be a good idea.

For the record, there was a short period of time in 2016 when the exact
fault address was delivered even when UFFD_FEATURE_EXACT_ADDRESS was not
provided. We had some arguments whether this was a regression...

BTW: I should have thought on the use-case of knowing the exact address
in huge-pages. It would have shorten my discussions with Andrea on whether
this feature (UFFD_FEATURE_EXACT_ADDRESS) is needed. :)