Re: [PATCH] tools/testing/selftests/vm/mlock2-tests: fix mlock2 false-negative errors

[Rafael Aquini]

On Mon, Mar 23, 2020 at 08:52:08AM +0100, Michal Hocko wrote:
> On Sun 22-03-20 09:36:49, Shakeel Butt wrote:
> > On Sat, Mar 21, 2020 at 9:31 PM Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > On Sat, 21 Mar 2020 22:03:26 -0400 Rafael Aquini <aquini@xxxxxxxxxx> wrote:
> > >
> > > > > > + * In order to sort out that race, and get the after fault checks consistent,
> > > > > > + * the "quick and dirty" trick below is required in order to force a call to
> > > > > > + * lru_add_drain_all() to get the recently MLOCK_ONFAULT pages moved to
> > > > > > + * the unevictable LRU, as expected by the checks in this selftest.
> > > > > > + */
> > > > > > +static void force_lru_add_drain_all(void)
> > > > > > +{
> > > > > > + sched_yield();
> > > > > > + system("echo 1 > /proc/sys/vm/compact_memory");
> > > > > > +}
> > > > >
> > > > > What is the sched_yield() for?
> > > > >
> > > >
> > > > Mostly it's there to provide a sleeping gap after the fault, whithout
> > > > actually adding an arbitrary value with usleep().
> > > >
> > > > It's not a hard requirement, but, in some of the tests I performed
> > > > (whithout that sleeping gap) I would still see around 1% chance
> > > > of hitting the false-negative. After adding it I could not hit
> > > > the issue anymore.
> > >
> > > It's concerning that such deep machinery as pagevec draining is visible
> > > to userspace.
> > >
> > 
> > We already have other examples like memcg stats where the
> > optimizations like batching per-cpu stats collection exposes
> > differences to the userspace. I would not be that worried here.
> 
> Agreed! Tests should be more tolerant for counters imprecision.
> Unevictable LRU is an optimization and transition to that list is a
> matter of an internal implementation detail.
>
> > > I suppose that for consistency and correctness we should perform a
> > > drain prior to each read from /proc/*/pagemap.  Presumably this would
> > > be far too expensive.
> > >
> > > Is there any other way?  One such might be to make the MLOCK_ONFAULT
> > > pages bypass the lru_add_pvecs?
> > >
> > 
> > I would rather prefer to have something similar to
> > /proc/sys/vm/stat_refresh which drains the pagevecs.
> 
> No, please don't. Pagevecs draining is by far not the only batching
> scheme we use and an interface like this would promise users to
> effectivelly force flushing all of them.
> 
> Can we simply update the test to be more tolerant to imprecisions
> instead?
> 

I don't think, thouhg, that this particular test case can be entirely
reduced as "counter imprecison".

The reason I think this is a different beast, is that having the page
being flagged as PG_unevictable is expected part of the aftermath of
a mlock* call. This selftest is, IMO, correctly verifying that fact,
as it checks the functionality correctness.

The problem boils down to the fact that the page would immediately
be flagged as PG_unevictable after the mlock (under MCL_FUTURE|MCL_ONFAULT
semantics) call, and the test was expecting it, and commit 9c4e6b1a7027f
changed that by "delaying" that flag setting.

As I mentioned, too, there's nothing wrong with the delayed setting of 
PG_unevictable, we just need to compensate for that fact in this test,
which is what this patch is suggesting to do.
  
> -- 
> Michal Hocko
> SUSE Labs
>