Re: [RESEND PATCH 2/2] iommu/iova: allocate iova_rcache->depot dynamicly

[zhangzekun (A)]

在 2023/8/15 23:15, John Garry 写道:
> On 12/08/2023 08:18, zhangzekun (A) wrote:
> > 在 2023/8/11 22:14, John Garry 写道:
> > > On 11/08/2023 14:02, Zhang Zekun wrote:
> > > > In fio test with 4k,read,and allowed cpus to 0-255, we observe a 
> > > > performance
> > > > decrease of IOPS.
> > The normal IOPS
> > > What do you mean by normal IOPS? Describe this "normal" scenario.
> > Hi, John
> >
> > The reason why I think 1980K is normal is that I have test the 
> > iova_rache
> > hit rate with all iova size, the average iova cache hit rate can 
> > reach up to
>
> Sorry to say, but I still don't know what you mean by the terms 
> "normal" and "abnormal" here. Is "normal" prior to the drop in IOPS 
> which you mention, above? If so, at what time do this occur?
Hi, John

The decrease is first observe in high concurrency fio test based on 
openeulr kernel-5.10, and the IOPS is about 300~400K , which is quite 
abnormal for some low fio concurrency test can have more than 1000K 
IOPS. The frame graph show that iovas alloc from slow path alloc_iova() 
takes a high percentage, and I find out that current struct of 
iova_rcache could  behave poor with machine have 256 cores and with 
device driver does not free and alloc iova evenly in a heavy work load. 
After optimizing the iova_rcache, the IOPS come to more than 1900K IOPS 
on openeuler kernel-5.10. The mainline of linux kernel have the same 
problem on openeuler kernel-5.10, but the IOPS does not improve such a 
large percentage. I use the term "normal" and "abnormal" here, maybe 
these pair of words should be replaced by "before optimization" and 
"after optimization".

Thanks,
Zekun