Re: [PATCH v3 1/2] bio: limit bio max size

[Changheun Lee]

> On Tue, Jan 26, 2021 at 06:26:02AM +0000, Damien Le Moal wrote:
> > On 2021/01/26 15:07, Ming Lei wrote:
> > > On Tue, Jan 26, 2021 at 04:06:06AM +0000, Damien Le Moal wrote:
> > >> On 2021/01/26 12:58, Ming Lei wrote:
> > >>> On Tue, Jan 26, 2021 at 10:32:34AM +0900, Changheun Lee wrote:
> > >>>> bio size can grow up to 4GB when muli-page bvec is enabled.
> > >>>> but sometimes it would lead to inefficient behaviors.
> > >>>> in case of large chunk direct I/O, - 32MB chunk read in user space -
> > >>>> all pages for 32MB would be merged to a bio structure if the pages
> > >>>> physical addresses are contiguous. it makes some delay to submit
> > >>>> until merge complete. bio max size should be limited to a proper size.
> > >>>>
> > >>>> When 32MB chunk read with direct I/O option is coming from userspace,
> > >>>> kernel behavior is below now. it's timeline.
> > >>>>
> > >>>>  | bio merge for 32MB. total 8,192 pages are merged.
> > >>>>  | total elapsed time is over 2ms.
> > >>>>  |------------------ ... ----------------------->|
> > >>>>                                                  | 8,192 pages merged a bio.
> > >>>>                                                  | at this time, first bio submit is done.
> > >>>>                                                  | 1 bio is split to 32 read request and issue.
> > >>>>                                                  |--------------->
> > >>>>                                                   |--------------->
> > >>>>                                                    |--------------->
> > >>>>                                                               ......
> > >>>>                                                                    |--------------->
> > >>>>                                                                     |--------------->|
> > >>>>                           total 19ms elapsed to complete 32MB read done from device. |
> > >>>>
> > >>>> If bio max size is limited with 1MB, behavior is changed below.
> > >>>>
> > >>>>  | bio merge for 1MB. 256 pages are merged for each bio.
> > >>>>  | total 32 bio will be made.
> > >>>>  | total elapsed time is over 2ms. it's same.
> > >>>>  | but, first bio submit timing is fast. about 100us.
> > >>>>  |--->|--->|--->|---> ... -->|--->|--->|--->|--->|
> > >>>>       | 256 pages merged a bio.
> > >>>>       | at this time, first bio submit is done.
> > >>>>       | and 1 read request is issued for 1 bio.
> > >>>>       |--------------->
> > >>>>            |--------------->
> > >>>>                 |--------------->
> > >>>>                                       ......
> > >>>>                                                  |--------------->
> > >>>>                                                   |--------------->|
> > >>>>         total 17ms elapsed to complete 32MB read done from device. |
> > >>>>
> > >>>> As a result, read request issue timing is faster if bio max size is limited.
> > >>>> Current kernel behavior with multipage bvec, super large bio can be created.
> > >>>> And it lead to delay first I/O request issue.
> > >>>>
> > >>>> Signed-off-by: Changheun Lee <nanich.lee@xxxxxxxxxxx>
> > >>>> ---
> > >>>>  block/bio.c            | 17 ++++++++++++++++-
> > >>>>  include/linux/bio.h    |  4 +++-
> > >>>>  include/linux/blkdev.h |  3 +++
> > >>>>  3 files changed, 22 insertions(+), 2 deletions(-)
> > >>>>
> > >>>> diff --git a/block/bio.c b/block/bio.c
> > >>>> index 1f2cc1fbe283..ec0281889045 100644
> > >>>> --- a/block/bio.c
> > >>>> +++ b/block/bio.c
> > >>>> @@ -287,6 +287,21 @@ void bio_init(struct bio *bio, struct bio_vec *table,
> > >>>>  }
> > >>>>  EXPORT_SYMBOL(bio_init);
> > >>>>  
> > >>>> +unsigned int bio_max_size(struct bio *bio)
> > >>>> +{
> > >>>> +	struct request_queue *q;
> > >>>> +
> > >>>> +	if (!bio->bi_disk)
> > >>>> +		return UINT_MAX;
> > >>>> +
> > >>>> +	q = bio->bi_disk->queue;
> > >>>> +	if (!blk_queue_limit_bio_size(q))
> > >>>> +		return UINT_MAX;
> > >>>> +
> > >>>> +	return blk_queue_get_max_sectors(q, bio_op(bio)) << SECTOR_SHIFT;
> > >>>> +}
> > >>>> +EXPORT_SYMBOL(bio_max_size);
> > >>>> +
> > >>>>  /**
> > >>>>   * bio_reset - reinitialize a bio
> > >>>>   * @bio:	bio to reset
> > >>>> @@ -877,7 +892,7 @@ bool __bio_try_merge_page(struct bio *bio, struct page *page,
> > >>>>  		struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1];
> > >>>>  
> > >>>>  		if (page_is_mergeable(bv, page, len, off, same_page)) {
> > >>>> -			if (bio->bi_iter.bi_size > UINT_MAX - len) {
> > >>>> +			if (bio->bi_iter.bi_size > bio_max_size(bio) - len) {
> > >>>>  				*same_page = false;
> > >>>>  				return false;
> > >>>>  			}
> > >>>
> > >>> So far we don't need bio->bi_disk or bio->bi_bdev(will be changed in
> > >>> Christoph's patch) during adding page to bio, so there is null ptr
> > >>> refereance risk.
> > >>>
> > >>>> diff --git a/include/linux/bio.h b/include/linux/bio.h
> > >>>> index 1edda614f7ce..cdb134ca7bf5 100644
> > >>>> --- a/include/linux/bio.h
> > >>>> +++ b/include/linux/bio.h
> > >>>> @@ -100,6 +100,8 @@ static inline void *bio_data(struct bio *bio)
> > >>>>  	return NULL;
> > >>>>  }
> > >>>>  
> > >>>> +extern unsigned int bio_max_size(struct bio *);
> > >>>> +
> > >>>>  /**
> > >>>>   * bio_full - check if the bio is full
> > >>>>   * @bio:	bio to check
> > >>>> @@ -113,7 +115,7 @@ static inline bool bio_full(struct bio *bio, unsigned len)
> > >>>>  	if (bio->bi_vcnt >= bio->bi_max_vecs)
> > >>>>  		return true;
> > >>>>  
> > >>>> -	if (bio->bi_iter.bi_size > UINT_MAX - len)
> > >>>> +	if (bio->bi_iter.bi_size > bio_max_size(bio) - len)
> > >>>>  		return true;
> > >>>>  
> > >>>>  	return false;
> > >>>> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> > >>>> index f94ee3089e01..3aeab9e7e97b 100644
> > >>>> --- a/include/linux/blkdev.h
> > >>>> +++ b/include/linux/blkdev.h
> > >>>> @@ -621,6 +621,7 @@ struct request_queue {
> > >>>>  #define QUEUE_FLAG_RQ_ALLOC_TIME 27	/* record rq->alloc_time_ns */
> > >>>>  #define QUEUE_FLAG_HCTX_ACTIVE	28	/* at least one blk-mq hctx is active */
> > >>>>  #define QUEUE_FLAG_NOWAIT       29	/* device supports NOWAIT */
> > >>>> +#define QUEUE_FLAG_LIMIT_BIO_SIZE 30	/* limit bio size */
> > >>>>  
> > >>>>  #define QUEUE_FLAG_MQ_DEFAULT	((1 << QUEUE_FLAG_IO_STAT) |		\
> > >>>>  				 (1 << QUEUE_FLAG_SAME_COMP) |		\
> > >>>> @@ -667,6 +668,8 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q);
> > >>>>  #define blk_queue_fua(q)	test_bit(QUEUE_FLAG_FUA, &(q)->queue_flags)
> > >>>>  #define blk_queue_registered(q)	test_bit(QUEUE_FLAG_REGISTERED, &(q)->queue_flags)
> > >>>>  #define blk_queue_nowait(q)	test_bit(QUEUE_FLAG_NOWAIT, &(q)->queue_flags)
> > >>>> +#define blk_queue_limit_bio_size(q)	\
> > >>>> +	test_bit(QUEUE_FLAG_LIMIT_BIO_SIZE, &(q)->queue_flags)
> > >>>
> > >>> I don't think it is a good idea by adding queue flag for this purpose,
> > >>> since this case just needs to limit bio size for not delay bio submission
> > >>> too much, which is kind of logical thing, nothing to do with request queue.
> > >>>
> > >>> Just wondering why you don't take the following way:
> > >>>
> > >>>
> > >>> diff --git a/block/bio.c b/block/bio.c
> > >>> index 99040a7e6656..35852f7f47d4 100644
> > >>> --- a/block/bio.c
> > >>> +++ b/block/bio.c
> > >>> @@ -1081,7 +1081,7 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter)
> > >>>   * It's intended for direct IO, so doesn't do PSI tracking, the caller is
> > >>>   * responsible for setting BIO_WORKINGSET if necessary.
> > >>>   */
> > >>> -int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
> > >>> +int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, bool sync)
> > >>>  {
> > >>>  	int ret = 0;
> > >>>  
> > >>> @@ -1092,12 +1092,20 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
> > >>>  		bio_set_flag(bio, BIO_NO_PAGE_REF);
> > >>>  		return 0;
> > >>>  	} else {
> > >>> +		/*
> > >>> +		 * Don't add too many pages in case of sync dio for
> > >>> +		 * avoiding delay bio submission too much especially
> > >>> +		 * pinning user pages in memory isn't cheap.
> > >>> +		 */
> > >>> +		const unsigned int max_size = sync ? (1U << 12) : UINT_MAX;
> > >>
> > >> 4KB max bio size ? That is a little small :)
> > > 
> > > It should have been (1U << 20), :-(
> > 
> > Sounds better !
> > 
> > > 
> > >> In any case, I am not a fan of using an arbitrary value not related to the
> > >> actual device characteristics. Wouldn't it be better to us the device
> > >> max_sectors limit ? And that limit would need to be zone_append_max_sectors for
> > >> zone append writes. So some helper like Changheun bio_max_size() may be useful.
> > > 
> > > Firstly, bio->bi_disk may not be initialized when adding page to bio; secondly this
> > > limit isn't really related with device, is it? Also if it is one queue limit, it has
> > > to be stacked.
> > 
> > 1MB can be used as a fallback default if the gendisk is not yet set. If it is,
> 
> IMO, only sync dio on slow machine needs such limit because pinning userspace
> pages to memory may take a bit long, so far not see other workloads needs this limit.
> 
> Even today I get queries from client about why 4MB user space IO won't be converted to
> 4MB bio, some workload needs big size IO.
> 

IMO, your solution is good. But I think it's a scenario specific solution.
Current approach could be better to remove bio submission delay basically.
And this is a option to enable in runtime by queue flag, default is no
limit of bio size. I think this solution affacts little on your work.

> > using a queue limit that does not cause bio splitting after submit makes most
> > sense as that avoid useless overhead. I agree, it is not critical, but it would
> > be nice to have some number that causes less splitting than the arbitrary 1MB.
> 
> That is another story. Each fs bio needs two allocation(one fixed length
> bio allocation, and variable length of bvec table), however bio splitting just
> needs single fixed length bio allocation. So if the source bio(fs bio) for holding
> data becomes smaller, splitting may become less, but more fs bio and bvec table
> allocation may be involved, not sure this way always gets better performance.
> 
> Also in theory, bio splitting may not need to allocate one whole bio
> allocation, what matters is just the actual position/size info of the
> to-be-splitted bio.
> 
> > E,g, most HDDs will likely have that 1MB BIO split... And max_sectors is a
> > stacked queue limit, no ? We could use max_hw_sectors too I think.
> 
> bio_add_page() is really fast path, and checking queue limit here may
> hurt performance because queue_limits reference is added to the fast path.
> 

Your concern point is good I don't like it too. But it'll be better than
adding new variable in bio structure I think.
Actually adding new check condition is not good itself, But queue flag
checking load is smaller than many condition checks in page_is_mergeable(). :)

> 
> 
> Thanks,
> Ming

---
Changheun Lee
Samsung Electronics