Re: [PATCH/RFC] z3fold: use per-page read/write lock

[Peter Zijlstra]

On Sat, Nov 05, 2016 at 02:49:46PM +0100, Vitaly Wool wrote:
> +/* Read-lock a z3fold page */
> +static void z3fold_page_rlock(struct z3fold_header *zhdr)
> +{
> +	while (!atomic_add_unless(&zhdr->page_lock, 1, Z3FOLD_PAGE_WRITE_FLAG))
> +		cpu_relax();
> +	smp_mb();
> +}
> +
> +/* Read-unlock a z3fold page */
> +static void z3fold_page_runlock(struct z3fold_header *zhdr)
> +{
> +	atomic_dec(&zhdr->page_lock);
> +	smp_mb();
> +}
> +
> +/* Write-lock a z3fold page */
> +static void z3fold_page_wlock(struct z3fold_header *zhdr)
> +{
> +	while (atomic_cmpxchg(&zhdr->page_lock, 0, Z3FOLD_PAGE_WRITE_FLAG) != 0)
> +		cpu_relax();
> +	smp_mb();
> +}
> +
> +/* Write-unlock a z3fold page */
> +static void z3fold_page_wunlock(struct z3fold_header *zhdr)
> +{
> +	atomic_sub(Z3FOLD_PAGE_WRITE_FLAG, &zhdr->page_lock);
> +	smp_mb();
> +}

This is trivially broken. What Andi said, don't roll your own locks.


The unlocks want: smp_mb__before_atomic() _before_ the atomic for
'obvious' reasons.

Also, this lock has serious starvation issues and is reader biased.