Re: [patch] mm, oom: stop reclaiming if GFP_ATOMIC will start failing soon

[Andrew Morton]

On Mon, 27 Apr 2020 16:03:56 -0700 (PDT) David Rientjes <rientjes@xxxxxxxxxx> wrote:

> On Mon, 27 Apr 2020, Andrew Morton wrote:
> 
> > > No - that would actually make the problem worse.
> > > 
> > > Today, per-zone min watermarks dictate when user allocations will loop or 
> > > oom kill.  should_reclaim_retry() currently loops if reclaim has succeeded 
> > > in the past few tries and we should be able to allocate if we are able to 
> > > reclaim the amount of memory that we think we can.
> > > 
> > > The issue is that this supposes that looping to reclaim more will result 
> > > in more free memory.  That doesn't always happen if there are concurrent 
> > > memory allocators.
> > > 
> > > GFP_ATOMIC allocators can access below these per-zone watermarks.  So the 
> > > issue is that per-zone free pages stays between ALLOC_HIGH watermarks 
> > > (the watermark that GFP_ATOMIC allocators can allocate to) and min 
> > > watermarks.  We never reclaim enough memory to get back to min watermarks 
> > > because reclaim cannot keep up with the amount of GFP_ATOMIC allocations.
> > 
> > But there should be an upper bound upon the total amount of in-flight
> > GFP_ATOMIC memory at any point in time?  These aren't like pagecache
> > which will take more if we give it more.  Setting the various
> > thresholds appropriately should ensure that blockable allocations don't
> > get their memory stolen by GPP_ATOMIC allocations?
> > 
> 
> Certainly if that upper bound is defined and enforced somewhere we would 
> not have run into this issue causing all userspace to become completely 
> unresponsive.  Do you have links to patches that proposed enforcing this 
> upper bound?

There is no such enforcement and there are no such patches, as I'm sure
you know.

No consumer of GFP_ATOMIC memory should consume an unbounded amount of
it.  Subsystems such as networking will consume a certain amount and
will then start recycling it.  The total amount in-flight will vary
over the longer term as workloads change.  A dynamically tuning
threshold system will need to adapt rapidly enough to sudden load
shifts, which might require unreasonable amounts of headroom.

Michal asked relevant questions regarding watermark tuning - an ansewr
to those would be interesting.  To amplify that, is it possible to
manually tune this system so that the problem no longer exhibits?  If
so, then why can't that tuning be performed automatically?