Re: [External] Re: [RFC PATCH] blk-throtl: Introduce sync queue for write ios

[hanjinke]

在 2022/12/13 上午6:40, Tejun Heo 写道:
> On Wed, Dec 07, 2022 at 12:38:26AM +0800, Jinke Han wrote:
> > From: Jinke Han <hanjinke.666@xxxxxxxxxxxxx>
> >
> > Now we don't distinguish sync write ios from normal buffer write ios
> > in blk-throtl. A bio with REQ_SYNC tagged always mean it will be wait
> > until write completion soon after it submit. So it's reasonable for sync
> > io to complete as soon as possible.
> >
> > In our test, fio writes a 100g file in sequential 4k blocksize in
> > a container with low bps limit configured (wbps=10M). More than 1200
> > ios were throttled in blk-throtl queue and the avarage throtle time
> > of each io is 140s. At the same time, the operation of saving a small
> > file by vim will be blocked amolst 140s. As a fsync will be send by vim,
> > the sync ios of fsync will be blocked by a huge amount of buffer write
> > ios ahead. This is also a priority inversion problem within one cgroup.
> > In the database scene, things got really bad with blk-throtle enabled
> > as fsync is called very often.
> >
> > This patch introduces a independent sync queue for write ios and gives
> > a huge priority to sync write ios. I think it's a nice respond to the
> > semantics of REQ_SYNC. Bios with REQ_META and REQ_PRIO gains the same
> > priority as they are important to fs. This may avoid some potential
> > priority inversion problems.
>
> I think the idea makes sense but wonder whether the implementation would be
> cleaner / simpler if the sq->queued[] are indexed by SYNC, ASYNC and the
> sync writes are queued in the sync queue together with reads.
>
> Thanks.
Should we keep the main category of io based READ and WRITE, and within 
READ / WRITE the subcategory were SYNC and ASYNC ? This may give less 
intrusion into existing frameworks.

Thanks.