Re: [PATCH 0/13] Parallel struct page initialisation v4
From: Waiman Long
Date: Tue May 05 2015 - 21:21:35 EST
On 05/05/2015 04:02 PM, Andrew Morton wrote:
On Tue, 5 May 2015 11:45:14 +0100 Mel Gorman<mgorman@xxxxxxx> wrote:
On Mon, May 04, 2015 at 02:30:46PM -0700, Andrew Morton wrote:
Before the patch, the boot time from elilo prompt to ssh login was 694s.
After the patch, the boot up time was 346s, a saving of 348s (about 50%).
Having to guesstimate the amount of memory which is needed for a
successful boot will be painful. Any number we choose will be wrong
99% of the time.
If the kswapd threads have started, all we need to do is to wait: take
a little nap in the allocator's page==NULL slowpath.
I'm not seeing any reason why we can't start kswapd much earlier -
right at the start of do_basic_setup()?
It doesn't even have to be kswapd, it just should be a thread pinned to
a done. The difficulty is that dealing with the system hashes means the
initialisation has to happen before vfs_caches_init_early() when there is
no scheduler.
I bet we can run vfs_caches_init_early() after sched_init(). Might
need a few little fixups.
Those allocations could be delayed further but then there is
the possibility that the allocations would not be contiguous and they'd
have to rely on CMA to make the attempt. That potentially alters the
performance of the large system hashes at run time.
hm, why. If the kswapd threads are running and busily creating free
pages then alloc_pages(order=10) can detect this situation and stall
for a while, waiting for kswapd to create an order-10 page.
Alternatively, the page allocator can go off and synchronously
initialize some pageframes itself. Keep doing that until the
allocation attempt succeeds.
Such an approach is much more robust than trying to predict how much
memory will be needed.
Most of those hash tables are allocated before smp_boot. In UP mode, you
can't have another thread initializing memory. So we really need to
preallocate enough for those tables.
Cheers,
Longman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/