Re: [RFC PATCH] percpu_ref: Make percpu_ref_tryget*() ACQUIRE operations

[Paul E. McKenney]

On Tue, Apr 13, 2021 at 10:47:03AM +0800, Huang Ying wrote:
> One typical use case of percpu_ref_tryget() family functions is as
> follows,
> 
>   if (percpu_ref_tryget(&p->ref)) {
> 	  /* Operate on the other fields of *p */
>   }
> 
> The refcount needs to be checked before operating on the other fields
> of the data structure (*p), otherwise, the values gotten from the
> other fields may be invalid or inconsistent.  To guarantee the correct
> memory ordering, percpu_ref_tryget*() needs to be the ACQUIRE
> operations.

I am not seeing the need for this.

If __ref_is_percpu() returns true, then the overall count must be non-zero
and there will be an RCU grace period between now and the time that this
count becomes zero.  For the calls to __ref_is_percpu() enclosed within
rcu_read_lock() and rcu_read_unlock(), the grace period will provide
the needed ordering.  (See the comment header for the synchronize_rcu()
function.)

Otherwise, when __ref_is_percpu() returns false, its caller does a
value-returning atomic read-modify-write operation, which provides
full ordering.

Either way, the required acquire semantics (and more) are already
provided, and in particular, this analysis covers the percpu_ref_tryget()
you call out above.

Or am I missing something subtle here?

							Thanx, Paul

> This function implements that via using smp_load_acquire() in
> __ref_is_percpu() to read the percpu pointer.
> 
> Signed-off-by: "Huang, Ying" <ying.huang@xxxxxxxxx>
> Cc: Tejun Heo <tj@xxxxxxxxxx>
> Cc: Kent Overstreet <kent.overstreet@xxxxxxxxx>
> Cc: "Paul E. McKenney" <paulmck@xxxxxxxxxx>
> Cc: Roman Gushchin <guro@xxxxxx>
> Cc: Ming Lei <ming.lei@xxxxxxxxxx>
> Cc: Al Viro <viro@xxxxxxxxxxxxxxxxxx>
> Cc: Miaohe Lin <linmiaohe@xxxxxxxxxx>
> ---
>  include/linux/percpu-refcount.h | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refcount.h
> index 16c35a728b4c..9838f7ea4bf1 100644
> --- a/include/linux/percpu-refcount.h
> +++ b/include/linux/percpu-refcount.h
> @@ -165,13 +165,13 @@ static inline bool __ref_is_percpu(struct percpu_ref *ref,
>  	 * !__PERCPU_REF_ATOMIC, which may be set asynchronously, and then
>  	 * used as a pointer.  If the compiler generates a separate fetch
>  	 * when using it as a pointer, __PERCPU_REF_ATOMIC may be set in
> -	 * between contaminating the pointer value, meaning that
> -	 * READ_ONCE() is required when fetching it.
> +	 * between contaminating the pointer value, smp_load_acquire()
> +	 * will prevent this.
>  	 *
> -	 * The dependency ordering from the READ_ONCE() pairs
> +	 * The dependency ordering from the smp_load_acquire() pairs
>  	 * with smp_store_release() in __percpu_ref_switch_to_percpu().
>  	 */
> -	percpu_ptr = READ_ONCE(ref->percpu_count_ptr);
> +	percpu_ptr = smp_load_acquire(&ref->percpu_count_ptr);
>  
>  	/*
>  	 * Theoretically, the following could test just ATOMIC; however,
> @@ -231,6 +231,9 @@ static inline void percpu_ref_get(struct percpu_ref *ref)
>   * Returns %true on success; %false on failure.
>   *
>   * This function is safe to call as long as @ref is between init and exit.
> + *
> + * This function is an ACQUIRE operation, that is, all memory operations
> + * after will appear to happen after checking the refcount.
>   */
>  static inline bool percpu_ref_tryget_many(struct percpu_ref *ref,
>  					  unsigned long nr)
> @@ -260,6 +263,9 @@ static inline bool percpu_ref_tryget_many(struct percpu_ref *ref,
>   * Returns %true on success; %false on failure.
>   *
>   * This function is safe to call as long as @ref is between init and exit.
> + *
> + * This function is an ACQUIRE operation, that is, all memory operations
> + * after will appear to happen after checking the refcount.
>   */
>  static inline bool percpu_ref_tryget(struct percpu_ref *ref)
>  {
> @@ -280,6 +286,9 @@ static inline bool percpu_ref_tryget(struct percpu_ref *ref)
>   * percpu_ref_tryget_live().
>   *
>   * This function is safe to call as long as @ref is between init and exit.
> + *
> + * This function is an ACQUIRE operation, that is, all memory operations
> + * after will appear to happen after checking the refcount.
>   */
>  static inline bool percpu_ref_tryget_live(struct percpu_ref *ref)
>  {
> -- 
> 2.30.2
>