Re: [PATCH v8 00/18] blk-mq/scsi: Provide hostwide shared tags for SCSI HBAs

[Hannes Reinecke]

On 8/19/20 5:20 PM, John Garry wrote:
> Hi all,
> 
> Here is v8 of the patchset.
> 
> In this version of the series, we keep the shared sbitmap for driver tags,
> and introduce changes to fix up the tag budgeting across request queues.
> We also have a change to count requests per-hctx for when an elevator is
> enabled, as an optimisation. I also dropped the debugfs changes - more on
> that below.
> 
> Some performance figures:
> 
> Using 12x SAS SSDs on hisi_sas v3 hw. mq-deadline results are included,
> but it is not always an appropriate scheduler to use.
> 
> Tag depth 		4000 (default)			260**
> 
> Baseline (v5.9-rc1):
> none sched:		2094K IOPS			513K
> mq-deadline sched:	2145K IOPS			1336K
> 
> Final, host_tagset=0 in LLDD *, ***:
> none sched:		2120K IOPS			550K
> mq-deadline sched:	2121K IOPS			1309K
> 
> Final ***:
> none sched:		2132K IOPS			1185			
> mq-deadline sched:	2145K IOPS			2097	
> 
> * this is relevant as this is the performance in supporting but not
>   enabling the feature
> ** depth=260 is relevant as some point where we are regularly waiting for
>    tags to be available. Figures were are a bit unstable here.
> *** Included "[PATCH V4] scsi: core: only re-run queue in
>     scsi_end_request() if device queue is busy"
> 
> A copy of the patches can be found here:
> https://github.com/hisilicon/kernel-dev/tree/private-topic-blk-mq-shared-tags-v8
> 
> The hpsa patch depends on:
> https://lore.kernel.org/linux-scsi/20200430131904.5847-1-hare@xxxxxxx/
> 
> And the smartpqi patch is not to be accepted.
> 
> Comments (and testing) welcome, thanks!
> 
> Differences to v7:
> - Add null_blk and scsi_debug support
> - Drop debugfs tags patch - it's too difficult to be the same between
> hostwide and non-hostwide, as discussed:
> https://lore.kernel.org/linux-scsi/1591810159-240929-1-git-send-email-john.garry@xxxxxxxxxx/T/#mb3eb462d8be40273718505989abd12f8228c15fd
> And from commit 6bf0eb550452 ("sbitmap: Consider cleared bits in
> sbitmap_bitmap_show()"), I guess not many used this anyway...
> - Add elevator per-hctx request count for optimisation
> - Break up "blk-mq: rename blk_mq_update_tag_set_depth()" into 2x patches
> - Pass flags for avoid per-hq queue tags init/free for hostwide tags
> - Add Don's reviewed-tag and tested-by tags to appropiate patches
> 	- (@Don, please let me know if issue with how I did this)
> - Add "scsi: core: Show nr_hw_queues in sysfs"
> - Rework megaraid SAS patch to have module param (Kashyap)
> - rebase
> 
> V7 is here for more info:
> https://lore.kernel.org/linux-scsi/1591810159-240929-1-git-send-email-john.garry@xxxxxxxxxx/T/#t
> 
> Hannes Reinecke (5):
>   blk-mq: Rename blk_mq_update_tag_set_depth()
>   blk-mq: Free tags in blk_mq_init_tags() upon error
>   scsi: Add host and host template flag 'host_tagset'
>   hpsa: enable host_tagset and switch to MQ
>   smartpqi: enable host tagset
> 
> John Garry (10):
>   blk-mq: Pass flags for tag init/free
>   blk-mq: Use pointers for blk_mq_tags bitmap tags
>   blk-mq: Facilitate a shared sbitmap per tagset
>   blk-mq: Relocate hctx_may_queue()
>   blk-mq: Record nr_active_requests per queue for when using shared
>     sbitmap
>   blk-mq: Record active_queues_shared_sbitmap per tag_set for when using
>     shared sbitmap
>   null_blk: Support shared tag bitmap
>   scsi: core: Show nr_hw_queues in sysfs
>   scsi: hisi_sas: Switch v3 hw to MQ
>   scsi: scsi_debug: Support host tagset
> 
> Kashyap Desai (2):
>   blk-mq, elevator: Count requests per hctx to improve performance
>   scsi: megaraid_sas: Added support for shared host tagset for
>     cpuhotplug
> 
> Ming Lei (1):
>   blk-mq: Rename BLK_MQ_F_TAG_SHARED as BLK_MQ_F_TAG_QUEUE_SHARED
> 
Now that Jens merged the block bits in his tree, wouldn't it be better
to re-send the SCSI bits only, thereby avoiding a potential merge error
later on?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		           Kernel Storage Architect
hare@xxxxxxx			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Felix Imendörffer