Re: [PATCH] percpu-refcount: relax limit on percpu_ref_reinit()

[Ming Lei]

On Tue, Sep 11, 2018 at 12:40:36PM +0800, jianchao.wang wrote:
> Hi Ming
> 
> On 09/11/2018 12:03 PM, Ming Lei wrote:
> > After the refcount is switched to atomic mode, the whole counting is
> > done on the atomic part. Then if the refcount need to switch to percpu mode
> > again, all percpu part of the counter is re-initialized as zero simply. This
> > is invariant with/without this patch.
> 
> Does the "whole counting" here means ?
> 
> (long)(sum (every cpu's refcounter)) + atomic refcounter

No.

We only check the 'whole counting' at atomic mode, so it is the atomic
part('ref->count'), please see percpu_ref_put_many(), in which ref->release()
is only called at atomic mode.

At percpu mode, the ref-count is only decreased/increased on the local CPU.

> 
> and when switch to atomic mode, there could be value left in atomic refcounter.
> then the unsigned long percpu refcounter cound be decreased from 0.
> 	
> From another angle, one request could be ended on a different cpu from the one where
> it is issued.

Right.

But,

In the fast path(either completion or issue path), the .q_usage_counter is only
increased/decreased, which can be done in either atomic or percpu mode.

And when we want to check the queued/inflight requests, the .q_usage_counter has
to be killed first by switching to atomic mode, such as in all kinds of
uses of blk_freeze_queue_start().


Thanks,
Ming