Re: [RFC PATCH 3/4] mm: madvise: implement lightweight guard page mechanism

[Lorenzo Stoakes]

On Mon, Oct 14, 2024 at 08:14:26PM +0200, Jann Horn wrote:
> On Mon, Oct 14, 2024 at 7:02 PM Lorenzo Stoakes
> <lorenzo.stoakes@xxxxxxxxxx> wrote:
> > On Mon, Oct 14, 2024 at 05:56:50PM +0200, Jann Horn wrote:
> > > On Mon, Oct 14, 2024 at 1:09 PM Lorenzo Stoakes <lorenzo.stoakes@xxxxxxxxxx> wrote:
> > > > On Fri, Oct 11, 2024 at 08:11:36PM +0200, Jann Horn wrote:
> > > > > On Fri, Sep 27, 2024 at 2:51 PM Lorenzo Stoakes <lorenzo.stoakes@xxxxxxxxxx> wrote:
> > > > By being optimistic and simply having the user having to handle looping
> > > > which seems reasonable (again, it's weird if you're installing poison
> > > > markers and another thread could be racing you) we avoid all of that.
> > >
> > > I guess one case in which that could happen legitimately is if you
> > > race a MADV_POISON on the area 0x1ff000-0x200100 (first page is
> > > populated, second page is not, pmd entry corresponding to 0x200000 is
> > > clear) with a page fault at 0x200200? So you could have a scenario
> > > like:
> > >
> > > 1. MADV_POISON starts walk_page_range()
> > > 2. MADV_POISON sees non-zero, non-poison PTE at 0x1ff000, stops the walk
> > > 3. MADV_POISON does zap_page_range_single()
> > > 4. pagefault at 0x200200 happens and populates with a hugepage
> > > 5. MADV_POISON enters walk_page_range()
> > > 6. MADV_POISON splits the THP
> > > 7. MADV_POISON sees a populated PTE
> >
> > You really shouldn't be seeing page faults in the range you are setting up
> > poison markers for _at all_ :) it's something you'd do ahead of time.
>
> But that's not what happens in my example - the address where the
> fault happens (0x200200) *is not* in the address range that
> MADV_POISON is called on (0x1ff000-0x200100). The fault and the
> MADV_POISON are in different 4KiB pages. What causes the conflict is
> that the fault and the MADV_POISON overlap the same *2MiB region*
> (both are in the region 0x200000-0x400000), and so THP stuff can
> effectively cause "page faults in the range you are setting up poison
> markers for".

Right sorry maybe I wasn't clear in what I said - there should not be
faults in the 'vicinity' of the poison pages, that is the range which you
potentially intend to protect with the poison markers.

HOWEVER this is problematic clearly for something like a userspace
allocator where you might be allocating small ranges that might fit within
a huge page.

At the same time, I think you'd have to get pretty unlucky - you'd need to
have faulted in enough for a huge page to be collapsed by THP, immediately
adjacent to where you are installing this poison range, which spans
multiple adjacent pages for some reason over the 2 MiB boundary (I assume
you mean 0x201000 not 0x200100 btw :P).

Anyway I think this is moot as I am warming to the idea of us just looping
to be honest.

There's a limit to how much can be faulted in (i.e. everything), and we
hold the lock.

The user is _choosing_ to call this function and if there happens to be
enormously huge amounts of faulting going on then so be it.

>
> > But of course it's possible some scenario could arise like that, that's
> > what the EAGAIN is for.
> >
> > I just really don't want to get into a realm of trying to prove absolutely
> > under all circumstances that we can't go on forever in a loop like that.
>
> We can have a bailout on signal_pending() or something like that, and
> a cond_resched(). Then as far as I know, it won't really make a
> difference in behavior whether the loop is in the kernel or in
> userspace code that's following what the manpage tells it to do -
> either way, the program will loop until it either finishes its work or
> is interrupted by a signal, and either way it can get preempted.
> (Well, except under PREEMPT_NONE, but that is basically asking for
> long scheduling delays.)
>
> And we do have other codepaths that have to loop endlessly if they
> keep racing with page table updates the wrong way, though I guess
> those loops are not going to always scan over a large address range
> over and over again...
>
> Maybe something like this would be good enough, and mirror what you'd
> otherwise tell userspace to do?
>
>
> @@ -1598,6 +1598,7 @@ int do_madvise(struct mm_struct *mm, unsigned
> long start, size_t len_in, int beh
>                 return madvise_inject_error(behavior, start, start + len_in);
>  #endif
>
> +retry:
>         write = madvise_need_mmap_write(behavior);
>         if (write) {
>                 if (mmap_write_lock_killable(mm))
> @@ -1627,6 +1628,12 @@ int do_madvise(struct mm_struct *mm, unsigned
> long start, size_t len_in, int beh
>         else
>                 mmap_read_unlock(mm);
>
> +       if (error == <<<some special value>>>) {
> +               if (!signal_pending(current))
> +                       goto retry;
> +               error = -ERESTARTNOINTR;
> +       }
> +
>         return error;
>  }
>
> Buuut, heh, actually, I just realized: You could even omit this and
> simply replace -EINTR with -ERESTARTNOINTR in your code as the error

Interesting that that exists had no idea :)

I think I'd rather avoid it as it looks so specific and a lot more like
asking for trouble than simply looping.

> value, and then the kernel would automatically go back into the
> syscall for you after going through signal handing and such, without
> userspace noticing.
> https://lore.kernel.org/all/20121206220955.GZ4939@xxxxxxxxxxxxxxxxxx/
> has some explanation on how this works. Basically it tells the
> architecture's syscall entry code to move the userspace instruction
> pointer back to the syscall instruction, so as soon as execution
> returns to userspace, the first userspace instruction that executes
> will immediately re-do the syscall. That might be the easiest way,
> even if it is maybe a *little* bit of an API abuse to use this thing
> without having a pending signal...
>
>
> > If you drop the lock for contention then you up the risk of that, it just
> > feels dangerous.
> >
> > A userland program can however live with a 'if EAGAIN try again' situation.
> >
> > An alternative approach to this might be to try to take the VMA lock, but
> > given the fraught situation with locking elsewhere I wonder if we should.
> >
> > Also, you have to be realy unlucky with timing for this to happen, even in
> > the scenario you mention (where you'd have to be unlucky with alignment
> > too), unless you're _heavily_ page faulting in the range, either way a
> > userland loop checking EAGAIN doesn't seem unreasonable.
>
> Yes, we could do -EINTR and document that for userspace, and as long
> as everyone using this properly reads the documentation, it will be
> fine. Though I imagine that from the userspace programmer perspective
> that's a weird API design - as in, if this error code always means I
> have to try again, why can't the kernel do that internally. It's kind
> of leaking an implementation detail into the UAPI.

Overall I am warming to us just looping.

I mean it's hard to argue against this being 'surprising' behaviour - the
user expects to be able to install poison markers and for that to just be
applied regardless of faulting.

And it's not exactly a huge amount more effort for us to simply loop, I
just wanted to avoid it to avoid having to think about whether there's
cases that could result in an eternal loop...

We'd need a write lock to VMA lock the VMAs and prevent racing faults (or
in the case of non-VMA lock kernels to simplly prevent them) which adds to
contention issues arguably a lot more than simply looping under the read
lock.

Let me think about this but I think I will go ahead and try to add
something simple that loops + checks for pending fatal signals for the next
iteration of this series.