Re: [PATCH 7/8] wbt: add general throttling mechanism

From: Omar Sandoval
Date: Thu Sep 01 2016 - 16:57:22 EST


On Wed, Aug 31, 2016 at 11:05:50AM -0600, Jens Axboe wrote:
> We can hook this up to the block layer, to help throttle buffered
> writes. Or NFS can tap into it, to accomplish the same.
>
> wbt registers a few trace points that can be used to track what is
> happening in the system:
>
> wbt_lat: 259:0: latency 2446318
> wbt_stat: 259:0: rmean=2446318, rmin=2446318, rmax=2446318, rsamples=1,
> wmean=518866, wmin=15522, wmax=5330353, wsamples=57
> wbt_step: 259:0: step down: step=1, window=72727272, background=8, normal=16, max=32
>
> This shows a sync issue event (wbt_lat) that exceeded it's time. wbt_stat
> dumps the current read/write stats for that window, and wbt_step shows a
> step down event where we now scale back writes. Each trace includes the
> device, 259:0 in this case.
>
> Signed-off-by: Jens Axboe <axboe@xxxxxx>
> ---
> include/linux/wbt.h | 118 +++++++++
> include/trace/events/wbt.h | 122 ++++++++++
> lib/Kconfig | 4 +
> lib/Makefile | 1 +
> lib/wbt.c | 587 +++++++++++++++++++++++++++++++++++++++++++++
> 5 files changed, 832 insertions(+)
> create mode 100644 include/linux/wbt.h
> create mode 100644 include/trace/events/wbt.h
> create mode 100644 lib/wbt.c
>

[snip]

> diff --git a/lib/Kconfig b/lib/Kconfig
> index d79909dc01ec..5a65a1f91889 100644
> --- a/lib/Kconfig
> +++ b/lib/Kconfig
> @@ -550,4 +550,8 @@ config STACKDEPOT
> bool
> select STACKTRACE
>
> +config WBT
> + bool
> + select SCALE_BITMAP

Looks like this snuck in from your experiments to get this to work on
top of scale_bitmap?

[snip]

> +void __wbt_done(struct rq_wb *rwb)
> +{
> + int inflight, limit;
> +
> + inflight = atomic_dec_return(&rwb->inflight);
> +
> + /*
> + * wbt got disabled with IO in flight. Wake up any potential
> + * waiters, we don't have to do more than that.
> + */
> + if (unlikely(!rwb_enabled(rwb))) {
> + wake_up_all(&rwb->wait);
> + return;
> + }
> +
> + /*
> + * If the device does write back caching, drop further down
> + * before we wake people up.
> + */
> + if (rwb->wc && !atomic_read(&rwb->bdi->wb.dirty_sleeping))
> + limit = 0;
> + else
> + limit = rwb->wb_normal;
> +
> + /*
> + * Don't wake anyone up if we are above the normal limit.
> + */
> + if (inflight && inflight >= limit)
> + return;
> +
> + if (waitqueue_active(&rwb->wait)) {
> + int diff = limit - inflight;
> +
> + if (!inflight || diff >= rwb->wb_background / 2)
> + wake_up_nr(&rwb->wait, 1);

wake_up(&rwb->wait)?

--
Omar