Re: [PATCH] ocfs2: fix deadlock when creating quota file

[Joseph Qi]

Hi Heming,

On 2/24/26 4:48 PM, Heming Zhao wrote:
> syzbot detected a circular locking dependency. the scenarios:
> 
>         CPU0                    CPU1
>         ----                    ----
>    lock(&ocfs2_quota_ip_alloc_sem_key);
>                                 lock(&ocfs2_sysfile_lock_key[USER_QUOTA_SYSTEM_INODE]);
>                                 lock(&ocfs2_quota_ip_alloc_sem_key);
>    lock(&ocfs2_sysfile_lock_key[ORPHAN_DIR_SYSTEM_INODE]);
> 
> or:
>        CPU0                    CPU1
>        ----                    ----
>   lock(&ocfs2_quota_ip_alloc_sem_key);
>                                lock(&dquot->dq_lock);
>                                lock(&ocfs2_quota_ip_alloc_sem_key);
>   lock(&ocfs2_sysfile_lock_key[ORPHAN_DIR_SYSTEM_INODE]);
> 
> Following are the code paths for above scenarios:
> 
> path_openat
>  ocfs2_create
>   ocfs2_mknod
>   + ocfs2_reserve_new_inode
>   |  ocfs2_reserve_suballoc_bits
>   |   inode_lock(alloc_inode) //C0: hold INODE_ALLOC_SYSTEM_INODE
>   |    //at end of this func, ocfs2_free_alloc_context(inode_ac) calls inode_unlock
>   |
>   + ocfs2_get_init_inode
>      __dquot_initialize
>       dqget
>        ocfs2_acquire_dquot
>        + ocfs2_lock_global_qf
>        |  down_write(&OCFS2_I(oinfo->dqi_gqinode)->ip_alloc_sem)//A2:grabbing
>        + ocfs2_create_local_dquot
>           down_write(&OCFS2_I(lqinode)->ip_alloc_sem)//A3:grabbing
> 
> evict
>  ocfs2_evict_inode
>   ocfs2_delete_inode
>    ocfs2_wipe_inode
>     + inode_lock(orphan_dir_inode) //B0:hold
>     + ...
>     + ocfs2_remove_inode
>        inode_lock(inode_alloc_inode) //INODE_ALLOC_SYSTEM_INODE
>         down_write(&inode->i_rwsem) //C1:grabbing
> 
> generic_file_direct_write
>  ocfs2_direct_IO
>   __blockdev_direct_IO
>    dio_complete
>     ocfs2_dio_end_io
>      ocfs2_dio_end_io_write
>      + down_write(&oi->ip_alloc_sem) //A0:hold
>      + ocfs2_del_inode_from_orphan
>         inode_lock(orphan_dir_inode) //B1:grabbing
> 
> Root cause for the circular locking:
> 
> DIO completion path:
>  holds oi->ip_alloc_sem and is trying to acquire the orphan_dir_inode lock.
> 
> evict path:
>  holds the orphan_dir_inode lock and is trying to acquire the
>  inode_alloc_inode lock.
> 
> ocfs2_mknod path:
>  Holds the inode_alloc_inode lock (to allocate a new quota file) and is
>  blocked waiting for oi->ip_alloc_sem in ocfs2_acquire_dquot().
> 

When running into ocfs2_mknod->ocfs2_get_init_inode,
ocfs2_reserve_new_inode is finished and INODE_ALLOC_SYSTEM_INODE gets
unlocked in ocfs2_free_ac_resource at the end.
So I get confused about the above deadlock sequence.
Am I missing something?

Joseph

> How to fix:
> 
> Replace down_write() with down_write_trylock() in ocfs2_acquire_dquot().
> If acquiring oi->ip_alloc_sem fails, return -EBUSY to abort the file creation
> routine and break the deadlock.
> 
> Reported-by: syzbot+78359d5fbb04318c35e9@xxxxxxxxxxxxxxxxxxxxxxxxx
> Closes: https://syzkaller.appspot.com/bug?extid=78359d5fbb04318c35e9
> Signed-off-by: Heming Zhao <heming.zhao@xxxxxxxx>
> ---
>  fs/ocfs2/quota_global.c | 5 ++++-
>  fs/ocfs2/quota_local.c  | 5 ++++-
>  2 files changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/ocfs2/quota_global.c b/fs/ocfs2/quota_global.c
> index e85b1ccf81be..4321d8f59402 100644
> --- a/fs/ocfs2/quota_global.c
> +++ b/fs/ocfs2/quota_global.c
> @@ -311,7 +311,10 @@ int ocfs2_lock_global_qf(struct ocfs2_mem_dqinfo *oinfo, int ex)
>  	spin_unlock(&dq_data_lock);
>  	if (ex) {
>  		inode_lock(oinfo->dqi_gqinode);
> -		down_write(&OCFS2_I(oinfo->dqi_gqinode)->ip_alloc_sem);
> +		if (!down_write_trylock(&OCFS2_I(oinfo->dqi_gqinode)->ip_alloc_sem)) {
> +			inode_unlock(oinfo->dqi_gqinode);
> +			return -EBUSY;
> +		}
>  	} else {
>  		down_read(&OCFS2_I(oinfo->dqi_gqinode)->ip_alloc_sem);
>  	}
> diff --git a/fs/ocfs2/quota_local.c b/fs/ocfs2/quota_local.c
> index c4e0117d8977..e451f3d96037 100644
> --- a/fs/ocfs2/quota_local.c
> +++ b/fs/ocfs2/quota_local.c
> @@ -1224,7 +1224,10 @@ int ocfs2_create_local_dquot(struct dquot *dquot)
>  	int status;
>  	u64 pcount;
>  
> -	down_write(&OCFS2_I(lqinode)->ip_alloc_sem);
> +	if (!down_write_trylock(&OCFS2_I(lqinode)->ip_alloc_sem)) {
> +		status = -EBUSY;
> +		goto out;
> +	}
>  	chunk = ocfs2_find_free_entry(sb, type, &offset);
>  	if (!chunk) {
>  		chunk = ocfs2_extend_local_quota_file(sb, type, &offset);