[RFC PATCH] Block device bio throttling support [V2]

[Vivek Goyal]

Hi,

This is V2 of the bio throttling patches. Following are changes since V1.

- Move throttling hook point from __make_request() to __generic_make_request().
  This will make sure that now throttling will work any block device which
  instanciated a request queue and I don't have to modify each block driver.
  So now it should work with device mapper devices also. Thanks to Mike Snitzer
  for the suggestion.

- Renamed blkio.read_bps_device to blkio.throttle.read_bps_device. Similar
  changes were done for blkio.write_bps_device.

- Exported two new files blkio.throttle.io_serviced and
  blkio.throttle_io_service_bytes to export number of bio and number of bytes
  served on request queue as seen by throttling policy.

- Introduce a config option CONFIG_BLK_DEV_THROTTLING, so that all the
  throttling logic can be compiled out. 

Previous version of patches are available here.

[V1] http://lkml.org/lkml/2010/9/1/251

Overview
========
Currently CFQ provides the weight based proportional division of bandwidth.
People also have been looking at extending block IO controller to provide
throttling/max bandwidth control.

I have started to write the support for throttling in block layer on
request queue so that it can be used both for higher level logical
devices as well as leaf nodes. This patch is still work in progress but
I wanted to post it for early feedback.

Basically currently I have hooked into __generic_make_request() function to
check which cgroup bio belongs to and if it is exceeding the specified
BW rate. If no, thread can continue to dispatch bio as it is otherwise
bio is queued internally and dispatched later with the help of a worker
thread.

HOWTO
=====
- Make sure CONFIG_BLK_CGROUP=y and CONFIG_BLK_DEV_THROTTLING=y.

- Mount blkio controller
        mount -t cgroup -o blkio none /cgroup/blkio

- Specify a bandwidth rate on particular device for root group. The format
  for policy is "<major>:<minor>  <byes_per_second>".

        echo "8:16  1048576" > /cgroup/blkio/blkio.throttle.read_bps_device

  Above will put a limit of 1MB/second on reads happening for root group
  on device having major/minor number 8:16.

- Run dd to read a file and see if rate is throttled to 1MB/s or not.

       	# dd if=/mnt/common/zerofile of=/dev/null bs=4K count=1024 iflag=direct
        1024+0 records in
        1024+0 records out
        4194304 bytes (4.2 MB) copied, 4.0001 s, 1.0 MB/s

 Limits for writes can be put using blkio.throttle.write_bps_device file.

Open Issues
===========
- Do we need to provide additional queue congestion semantics as we are
  throttling and queuing bios at request queue and probably we don't want
  a user space application to consume all the memory allocating bios
  and bombarding request queue with those bios.

TODO
====
- Lots of testing, bug fixes.
- Provide support for enforcing limits in IOPS.

Any feedback is welcome.

Overall changes look as follows.

 Documentation/cgroups/blkio-controller.txt |   89 +++-
 block/Kconfig                              |   12 +
 block/Makefile                             |    1 +
 block/blk-cgroup.c                         |  699 ++++++++++++++++-----
 block/blk-cgroup.h                         |   75 +++-
 block/blk-core.c                           |   24 +
 block/blk-throttle.c                       |  946 ++++++++++++++++++++++++++++
 block/blk.h                                |    4 +-
 block/cfq-iosched.c                        |    4 +
 block/cfq.h                                |    2 +-
 include/linux/blk_types.h                  |    3 +
 include/linux/blkdev.h                     |   24 +
 init/Kconfig                               |    9 +-

Thanks
Vivek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/