[PATCHSET REPOST percpu/for-3.18] percpu: implement atomic allocation support

From: Tejun Heo
Date: Fri Aug 22 2014 - 12:53:35 EST


(the initial posting was missing cc's, reposting)

Due to the use of vmalloc area allocations and page table populations,
preparing percpu areas require GFP_KERNEL and all the allocator users
are expected to be able to perform GFP_KERNEL allocations. This is
mostly okay but there are cases where atomic percpu allocations are
necessary usually in the IO path.

Currently, blk-throttle is implementing its own ad-hoc async
allocation and there are some more planned similar usages. I posted
[1] percpu_pool which generalizes the percpu atomic allocation a bit
but this was a bit too cumbersome especially to use with other library
data structures which make use of percpu memory as a part of it.

This patchset implements proper atomic allocation support in the
percpu allocator. It's largely composed of two parts. The first is
updates to the area allocator so that it can skip non-populated areas.
The second is async filling mechanisms which try to maintain a certain
level of empty populated areas. The allocator currently tries to keep
the number of empty populated pages between 2 and 4. Even with fairly
aggressive back-to-back allocations, this seems enough to satisfy most
allocations as long as the allocation size is under a page.

This patchset contains the following 13 patches.

0001-percpu-remove-the-usage-of-separate-populated-bitmap.patch
0002-percpu-remove-may_alloc-from-pcpu_get_pages.patch
0003-percpu-move-common-parts-out-of-pcpu_-de-populate_ch.patch
0004-percpu-move-region-iterations-out-of-pcpu_-de-popula.patch
0005-percpu-make-percpu-km-set-chunk-populated-bitmap-pro.patch
0006-percpu-restructure-locking.patch
0007-percpu-make-pcpu_alloc_area-capable-of-allocating-on.patch
0008-percpu-indent-the-population-block-in-pcpu_alloc.patch
0009-percpu-implement-__-alloc_percpu_gfp.patch
0010-percpu-make-sure-chunk-map-array-has-available-space.patch
0011-percpu-implmeent-pcpu_nr_empty_pop_pages-and-chunk-n.patch
0012-percpu-rename-pcpu_reclaim_work-to-pcpu_balance_work.patch
0013-percpu-implement-asynchronous-chunk-population.patch

0001-0005 are prep patches.

0006 restructures locking so that populated areas can be given out w/o
grabbing the mutex.

0007-0009 implement alloc_percpu_gfp() which supports atomic
allocations.

0010-0013 implement async filling mechanisms.

This patchset is on top of percpu/for-3.17-fixes 849f5169097e
("percpu: perform tlb flush after pcpu_map_pages() failure") and is
available in the following git branch.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git review-atomic-alloc

include/linux/percpu.h | 13 -
mm/percpu-km.c | 16 -
mm/percpu-vm.c | 162 +++------------
mm/percpu.c | 523 +++++++++++++++++++++++++++++++++++++++----------
4 files changed, 479 insertions(+), 235 deletions(-)

Thanks.

--
tejun

[1] http://lkml.kernel.org/g/20140718200804.GG13012@xxxxxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/