Re: [BUG]Writeback Cgroup/Dirty Throttle: very small buffered write thoughput caused by writeback cgroup and dirty thottle

[Miao Xie]

Cc linux-kernel mail list

on 2016/5/11 at 22:15, Miao Xie wrote:
> Hi, Tejun and Fengguang
>
> I found that buffered write thoughput was dropped down by writeback cgroup and dirty thottle on
> 4.6-rc7 kernel. If I ran benchmark on the top block cgroup, the thoughput was more than 1500MB/s.
> If I ran benchmark on a new block cgroup, the thoughput was down to 4MB/s.
>
> Steps to reproduce:
> # mount -t cgroup2 cgroup <cgrp_mnt>
> # echo "+io +memory" > <cgrp_mnt>/cgroup.subtree_control
> # mkdir <cgrp_mnt>/aaa
> # echo $$ > <cgrp_mnt>/aaa/cgroup.procs
> # fio test.config
> job0: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
> fio-2.2.8
> Starting 1 thread
> Jobs: 1 (f=1): [W(1)] [3.7% done] [0KB/4000KB/0KB /s] [0/1000/0 iops] [eta 04m:50s]
>
> Fio configuration is:
> [global]
> bs=4K
> direct=0
> ioengine=psync
> iodepth=1
> directory=/mnt/ext4/tstdir0
> time_based
> runtime=300
> group_reporting
> size=16G
> sync=0
> max_latency=120000000
> thread
>
> [job0]
> numjobs=1
> rw=write
>
> My box has 48 cores and 188GB memory, but I set
> vm.dirty_background_bytes = 268435456
> vm.dirty_bytes = 536870912
>
> if I set vm.dirty_background_bytes and vm.dirty_bytes to be a large number(vm.dirty_background_bytes = 3GB,
> vm.dirty_bytes = 4GB), then fio thoughput would be more than 1500MB/s. and then if I reset them to the original
> value(the above ones), the thoughout would be down to 500MB/s.
>
> And according my debug, I found fio sleeped for 1ms every time we dirty a page(balance dirty pages) when
> the thoughput was down to 4MB/s, it might be a bug of dirty throttle when we open write back cgroup, I think.
>
> Tejun and Fengguang, please let me know what you guys think about this issue, and if you have
> any suggestions for possible solutions, Any input is greatly appreciated!
>
> Thanks
> Miao