[PATCH] refcount: provide same memory ordering guarantees as in atomic_t

[Elena Reshetova]

Currently arch. independent implementation of refcount_t in
lib/refcount.c provides weak memory ordering guarantees
compare to its analog atomic_t implementations.
While it should not be a problem for most of the actual
cases of refcounters, it is more understandable for everyone
(and more error-prone for future users) to provide exactly
same memory ordering guarantees as atomics.

If speed is of a concern, then either more efficient arch.
dependent refcount_t implementation should be used or if there
are enough users in the future we might need to provide both
strict and relaxed refcount_t APIs.

Suggested-by: Kees Cook <keescook@xxxxxxxxxxxx>
Signed-off-by: Elena Reshetova <elena.reshetova@xxxxxxxxx>
---
 lib/refcount.c | 71 +++++-----------------------------------------------------
 1 file changed, 6 insertions(+), 65 deletions(-)

diff --git a/lib/refcount.c b/lib/refcount.c
index 5d0582a..cc6946e 100644
--- a/lib/refcount.c
+++ b/lib/refcount.c
@@ -8,29 +8,7 @@
  * there. This avoids wrapping the counter and causing 'spurious'
  * use-after-free issues.
  *
- * Memory ordering rules are slightly relaxed wrt regular atomic_t functions
- * and provide only what is strictly required for refcounts.
- *
- * The increments are fully relaxed; these will not provide ordering. The
- * rationale is that whatever is used to obtain the object we're increasing the
- * reference count on will provide the ordering. For locked data structures,
- * its the lock acquire, for RCU/lockless data structures its the dependent
- * load.
- *
- * Do note that inc_not_zero() provides a control dependency which will order
- * future stores against the inc, this ensures we'll never modify the object
- * if we did not in fact acquire a reference.
- *
- * The decrements will provide release order, such that all the prior loads and
- * stores will be issued before, it also provides a control dependency, which
- * will order us against the subsequent free().
- *
- * The control dependency is against the load of the cmpxchg (ll/sc) that
- * succeeded. This means the stores aren't fully ordered, but this is fine
- * because the 1->0 transition indicates no concurrency.
- *
- * Note that the allocator is responsible for ordering things between free()
- * and alloc().
+ * Memory ordering rules are exactly the same as with regular atomic_t functions
  *
  */
 
@@ -46,10 +24,6 @@
  *
  * Will saturate at UINT_MAX and WARN.
  *
- * Provides no memory ordering, it is assumed the caller has guaranteed the
- * object memory to be stable (RCU, etc.). It does provide a control dependency
- * and thereby orders future stores. See the comment on top.
- *
  * Use of this function is not recommended for the normal reference counting
  * use case in which references are taken and released one at a time.  In these
  * cases, refcount_inc(), or one of its variants, should instead be used to
@@ -72,7 +46,7 @@ bool refcount_add_not_zero(unsigned int i, refcount_t *r)
 		if (new < val)
 			new = UINT_MAX;
 
-	} while (!atomic_try_cmpxchg_relaxed(&r->refs, &val, new));
+	} while (!atomic_try_cmpxchg(&r->refs, &val, new));
 
 	WARN_ONCE(new == UINT_MAX, "refcount_t: saturated; leaking memory.\n");
 
@@ -87,10 +61,6 @@ EXPORT_SYMBOL(refcount_add_not_zero);
  *
  * Similar to atomic_add(), but will saturate at UINT_MAX and WARN.
  *
- * Provides no memory ordering, it is assumed the caller has guaranteed the
- * object memory to be stable (RCU, etc.). It does provide a control dependency
- * and thereby orders future stores. See the comment on top.
- *
  * Use of this function is not recommended for the normal reference counting
  * use case in which references are taken and released one at a time.  In these
  * cases, refcount_inc(), or one of its variants, should instead be used to
@@ -108,10 +78,6 @@ EXPORT_SYMBOL(refcount_add);
  *
  * Similar to atomic_inc_not_zero(), but will saturate at UINT_MAX and WARN.
  *
- * Provides no memory ordering, it is assumed the caller has guaranteed the
- * object memory to be stable (RCU, etc.). It does provide a control dependency
- * and thereby orders future stores. See the comment on top.
- *
  * Return: true if the increment was successful, false otherwise
  */
 bool refcount_inc_not_zero(refcount_t *r)
@@ -127,7 +93,7 @@ bool refcount_inc_not_zero(refcount_t *r)
 		if (unlikely(!new))
 			return true;
 
-	} while (!atomic_try_cmpxchg_relaxed(&r->refs, &val, new));
+	} while (!atomic_try_cmpxchg(&r->refs, &val, new));
 
 	WARN_ONCE(new == UINT_MAX, "refcount_t: saturated; leaking memory.\n");
 
@@ -141,9 +107,6 @@ EXPORT_SYMBOL(refcount_inc_not_zero);
  *
  * Similar to atomic_inc(), but will saturate at UINT_MAX and WARN.
  *
- * Provides no memory ordering, it is assumed the caller already has a
- * reference on the object.
- *
  * Will WARN if the refcount is 0, as this represents a possible use-after-free
  * condition.
  */
@@ -162,10 +125,6 @@ EXPORT_SYMBOL(refcount_inc);
  * ultimately leak on underflow and will fail to decrement when saturated
  * at UINT_MAX.
  *
- * Provides release memory ordering, such that prior loads and stores are done
- * before, and provides a control dependency such that free() must come after.
- * See the comment on top.
- *
  * Use of this function is not recommended for the normal reference counting
  * use case in which references are taken and released one at a time.  In these
  * cases, refcount_dec(), or one of its variants, should instead be used to
@@ -187,7 +146,7 @@ bool refcount_sub_and_test(unsigned int i, refcount_t *r)
 			return false;
 		}
 
-	} while (!atomic_try_cmpxchg_release(&r->refs, &val, new));
+	} while (!atomic_try_cmpxchg(&r->refs, &val, new));
 
 	return !new;
 }
@@ -200,10 +159,6 @@ EXPORT_SYMBOL(refcount_sub_and_test);
  * Similar to atomic_dec_and_test(), it will WARN on underflow and fail to
  * decrement when saturated at UINT_MAX.
  *
- * Provides release memory ordering, such that prior loads and stores are done
- * before, and provides a control dependency such that free() must come after.
- * See the comment on top.
- *
  * Return: true if the resulting refcount is 0, false otherwise
  */
 bool refcount_dec_and_test(refcount_t *r)
@@ -218,9 +173,6 @@ EXPORT_SYMBOL(refcount_dec_and_test);
  *
  * Similar to atomic_dec(), it will WARN on underflow and fail to decrement
  * when saturated at UINT_MAX.
- *
- * Provides release memory ordering, such that prior loads and stores are done
- * before.
  */
 void refcount_dec(refcount_t *r)
 {
@@ -236,9 +188,6 @@ EXPORT_SYMBOL(refcount_dec);
  * No atomic_t counterpart, it attempts a 1 -> 0 transition and returns the
  * success thereof.
  *
- * Like all decrement operations, it provides release memory order and provides
- * a control dependency.
- *
  * It can be used like a try-delete operator; this explicit case is provided
  * and not cmpxchg in generic, because that would allow implementing unsafe
  * operations.
@@ -249,7 +198,7 @@ bool refcount_dec_if_one(refcount_t *r)
 {
 	int val = 1;
 
-	return atomic_try_cmpxchg_release(&r->refs, &val, 0);
+	return atomic_try_cmpxchg(&r->refs, &val, 0);
 }
 EXPORT_SYMBOL(refcount_dec_if_one);
 
@@ -281,7 +230,7 @@ bool refcount_dec_not_one(refcount_t *r)
 			return true;
 		}
 
-	} while (!atomic_try_cmpxchg_release(&r->refs, &val, new));
+	} while (!atomic_try_cmpxchg(&r->refs, &val, new));
 
 	return true;
 }
@@ -296,10 +245,6 @@ EXPORT_SYMBOL(refcount_dec_not_one);
  * Similar to atomic_dec_and_mutex_lock(), it will WARN on underflow and fail
  * to decrement when saturated at UINT_MAX.
  *
- * Provides release memory ordering, such that prior loads and stores are done
- * before, and provides a control dependency such that free() must come after.
- * See the comment on top.
- *
  * Return: true and hold mutex if able to decrement refcount to 0, false
  *         otherwise
  */
@@ -327,10 +272,6 @@ EXPORT_SYMBOL(refcount_dec_and_mutex_lock);
  * Similar to atomic_dec_and_lock(), it will WARN on underflow and fail to
  * decrement when saturated at UINT_MAX.
  *
- * Provides release memory ordering, such that prior loads and stores are done
- * before, and provides a control dependency such that free() must come after.
- * See the comment on top.
- *
  * Return: true and hold spinlock if able to decrement refcount to 0, false
  *         otherwise
  */
-- 
2.7.4