Re: [PATCH] vmscan: do not throttle kthreads due to too_many_isolated

[Vladimir Davydov]

On Wed, Nov 25, 2015 at 07:27:57PM +0300, Vladimir Davydov wrote:
> On Wed, Nov 25, 2015 at 04:45:13PM +0100, Vlastimil Babka wrote:
> > On 11/25/2015 04:36 PM, Vladimir Davydov wrote:
> > > Block device drivers often hand off io request processing to kernel
> > > threads (example: device mapper). If such a thread calls kmalloc, it can
> > > dive into direct reclaim path and end up waiting for too_many_isolated
> > > to return false, blocking writeback. This can lead to a dead lock if the
> > 
> > Shouldn't such allocation lack __GFP_IO to prevent this and other kinds of
> > deadlocks? And/or have mempools?
> 
> Not necessarily. loopback is an example: it can call
> grab_cache_write_begin -> add_to_page_cache_lru with GFP_KERNEL.

Anyway, kthreads that use GFP_NOIO and/or mempool aren't safe either,
because it isn't an allocation context problem: the reclaimer locks up
not because it tries to take an fs/io lock the caller holds, but because
it waits for isolated pages to be put back, which will never happen,
since processes that isolated them depend on the kthread making
progress. This is purely a reclaimer heuristic, which kmalloc users are
not aware of.

My point is that, in contrast to userspace processes, it is dangerous to
throttle kthreads in the reclaimer, because they might be responsible
for reclaimer progress (e.g. performing writeback).

Regarding side effects of this patch. Well, there aren't many kthreads
out there, so I don't believe this can put the system under the risk of
thrashing because of isolating too many reclaimable pages.

Thanks,
Vladimir

> 
> > PF_KTHREAD looks like a big hammer to me that will solve only one
> > potential problem...
> 
> This problem can result in processes hanging forever. Any ideas how this
> could be fixed in a better way?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/