[GIT PULL] Series of blkcg updates for 4.3

[Jens Axboe]

Hi Linus,

A bit later in the cycle, but this has been in the block tree for a
a while. This is basically four patchsets from Tejun, that improve our
buffered cgroup writeback. It was dependent on the other cgroup changes,
but they went in earlier in this cycle.

Series 1 is set of 5 patches that has cgroup writeback updates:

* bdi_writeback iteration fix which could lead to some wb's being
  skipped or repeated during e.g. sync under memory pressure.

* Simplification of wb work wait mechanism.

* Writeback tracepoints updated to report cgroup.

Series 2 is is a set of updates for the CFQ cgroup writeback handling:

* Switched from GFP_ATOMIC to GFP_NOWAIT as suggested by Jeff.

* Other misc review points addressed, acks added and rebased.

cfq has always charged all async IOs to the root cgroup.  It didn't
have much choice as writeback didn't know about cgroups and there was
no way to tell who to blame for a given writeback IO.  writeback
finally grew support for cgroups and now tags each writeback IO with
the appropriate cgroup to charge it against.

This patchset updates cfq so that it follows the blkcg each bio is
tagged with.  Async cfq_queues are now shared across cfq_group, which
is per-cgroup, instead of per-request_queue cfq_data.  This makes all
IOs follow the weight based IO resource distribution implemented by
cfq.

Series 3 is the blkcg policy cleanup patches:

This patchset contains assorted cleanups for blkcg_policy methods and
blk[c]g_policy_data handling.

* alloc/free added for blkg_policy_data.  exit dropped.

* alloc/free added for blkcg_policy_data.

* blk-throttle's async percpu allocation is replaced with direct
  allocation.

* all methods now take blk[c]g_policy_data instead of blkcg_gq or
  blkcg.

And finally, series 4 is a set of patches cleaning up the blkcg stats
handling:

* The following patches added to consolidate blkcg entry point and
blkg creation. This is in itself is an improvement and helps
colllecting common stats on bio issue.

* per-blkg stats now accounted on bio issue rather than request
completion so that bio based and request based drivers can behave
the same way. The issue was spotted by Vivek.

blkcg's stats have always been somwhat of a mess. This patchset tries
to improve the situation a bit.

* cfq-iosched implements custom recursive stats and blk-throttle
implements custom per-cpu stats. This patchset make blkcg core
support both by default.

* cfq-iosched and blk-throttle keep track of the same stats multiple
times. Unify them.


Please pull! There will be two trivial merge conflicts, in
linux/blk-cgroup.h and fs/fs-writeback.c, just a heads up.


  git://git.kernel.dk/linux-block.git for-4.3/blkcg

----------------------------------------------------------------
Jens Axboe (1):
      Merge branch 'for-4.3-unified-base' of git://git.kernel.org/.../tj/cgroup into for-4.3/blkcg

Tejun Heo (45):
      writeback: bdi_for_each_wb() iteration is memcg ID based not blkcg
      writeback: remove wb_writeback_work->single_wait/done
      writeback: explain why @inode is allowed to be NULL for inode_congested()
      kernfs: implement kernfs_path_len()
      writeback: update writeback tracepoints to report cgroup
      cfq-iosched: simplify control flow in cfq_get_queue()
      cfq-iosched: fix async oom queue handling
      cfq-iosched: fix oom cfq_queue ref leak in cfq_set_request()
      cfq-iosched: minor cleanups
      blkcg, cfq-iosched: use GFP_NOWAIT instead of GFP_ATOMIC for non-critical allocations
      cfq-iosched: remove @gfp_mask from cfq_find_alloc_queue()
      cfq-iosched: move cfq_group determination from cfq_find_alloc_queue() to cfq_get_queue()
      cfq-iosched: fold cfq_find_alloc_queue() into cfq_get_queue()
      cfq-iosched: charge async IOs to the appropriate blkcg's instead of the root
      blkcg: remove unnecessary request_list->blkg NULL test in blk_put_rl()
      blkcg: use blkg_free() in blkcg_init_queue() failure path
      blkcg: remove unnecessary blkcg_root handling from css_alloc/free paths
      blkcg: restructure blkg_policy_data allocation in blkcg_activate_policy()
      blkcg: make blkcg_activate_policy() allow NULL ->pd_init_fn
      blkcg: replace blkcg_policy->pd_size with ->pd_alloc/free_fn() methods
      blk-throttle: remove asynchrnous percpu stats allocation mechanism
      blk-throttle: clean up blkg_policy_data alloc/init/exit/free methods
      blkcg: make blkcg_policy methods take a pointer to blkcg_policy_data
      blkcg: minor updates around blkcg_policy_data
      blkcg: replace blkcg_policy->cpd_size with ->cpd_alloc/free_fn() methods
      blkcg: inline [__]blkg_lookup()
      blkcg: move root blkg lookup optimization from throtl_lookup_tg() to __blkg_lookup()
      blk-throttle: improve queue bypass handling
      blkcg: consolidate blkg creation in blkcg_bio_issue_check()
      blkcg: add blkg_[rw]stat->aux_cnt and replace cfq_group->dead_stats with it
      blkcg: make blkcg_[rw]stat per-cpu
      blkcg: make blkg_[rw]stat_recursive_sum() to be able to index into blkcg_gq
      blkcg: move io_service_bytes and io_serviced stats into blkcg_gq
      blkcg: remove cfqg_stats->sectors
      blkcg: reduce stack usage of blkg_rwstat_recursive_sum()
      blkcg: remove unnecessary NULL checks from __cfqg_set_weight_device()
      blkcg: refine error codes returned during blkcg configuration
      blkcg: rename subsystem name from blkio to io
      blkcg: mark existing cftypes as legacy
      blkcg: move body parsing from blkg_conf_prep() to its callers
      blkcg: separate out tg_conf_updated() from tg_set_conf()
      blkcg: misc preparations for unified hierarchy interface
      blkcg: implement interface for the unified hierarchy
      blkcg: s/CFQ_WEIGHT_*/CFQ_WEIGHT_LEGACY_*/
      blkcg: use CGROUP_WEIGHT_* scale for io.weight on the unified hierarchy

 Documentation/cgroups/blkio-controller.txt  |  24 +-
 Documentation/cgroups/unified-hierarchy.txt |  61 ++-
 block/bio.c                                 |   2 +-
 block/blk-cgroup.c                          | 524 ++++++++++++++--------
 block/blk-core.c                            |   4 +-
 block/blk-throttle.c                        | 505 +++++++++------------
 block/blk.h                                 |   5 -
 block/cfq-iosched.c                         | 651 +++++++++++++++-------------
 fs/fs-writeback.c                           | 147 ++-----
 fs/kernfs/dir.c                             |  23 +
 include/linux/backing-dev.h                 |  26 +-
 include/linux/blk-cgroup.h                  | 340 ++++++++++-----
 include/linux/cgroup_subsys.h               |   2 +-
 include/linux/kernfs.h                      |   4 +
 include/trace/events/writeback.h            | 180 ++++++--
 mm/backing-dev.c                            |   4 +-
 mm/page-writeback.c                         |   6 +-
 17 files changed, 1424 insertions(+), 1084 deletions(-)

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/