Re: [PATCH] ocfs2: Fix deadlock in ocfs2_get_system_file_inode

From: Joseph Qi
Date: Sun Sep 22 2024 - 22:52:17 EST




On 9/22/24 1:50 AM, Mohammed Anees wrote:
> syzbot has found a possible deadlock in ocfs2_get_system_file_inode [1].
>
> The scenario is depicted here,
>
> CPU0 CPU1
> lock(&ocfs2_file_ip_alloc_sem_key);
> lock(&osb->system_file_mutex);
> lock(&ocfs2_file_ip_alloc_sem_key);
> lock(&osb->system_file_mutex);
>
> The function calls which could lead to this are:
>
> CPU0
> ocfs2_write_begin - lock(&ocfs2_file_ip_alloc_sem_key);
> .


>From the report link, it's ocfs2_mknod(), but not
ocfs2_write_begin().

> .
> .
> ocfs2_get_system_file_inode - lock(&osb->system_file_mutex);
>
> CPU1 -
> ocfs2_get_system_file_inode - lock(&osb->system_file_mutex);

>From the report link, it is in the flow of ocfs2_fill_super().
I'm not sure how it actually happens since user has to mount ocfs2
before doing any operations, e.g. create a file.
Anyway, since many flows will call ocfs2_get_system_file_inode(),
so it will theoretically happen.

> .
> .
> .
> ocfs2_read_virt_blocks - lock(&ocfs2_file_ip_alloc_sem_key);
>
> This issue can be resolved by making the down_read -> down_read_try
> in the ocfs2_read_virt_blocks.
>
> [1] https://syzkaller.appspot.com/bug?extid=e0055ea09f1f5e6fabdd
>
> Reported-and-tested-by: syzbot+e0055ea09f1f5e6fabdd@xxxxxxxxxxxxxxxxxxxxxxxxx
> Closes: https://syzkaller.appspot.com/bug?extid=e0055ea09f1f5e6fabdd
> Signed-off-by: Mohammed Anees <pvmohammedanees2003@xxxxxxxxx>
> ---
> fs/ocfs2/extent_map.c | 16 +++++++++++++++-
> 1 file changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/fs/ocfs2/extent_map.c b/fs/ocfs2/extent_map.c
> index 70a768b62..f83d0a3b6 100644
> --- a/fs/ocfs2/extent_map.c
> +++ b/fs/ocfs2/extent_map.c
> @@ -12,6 +12,7 @@
> #include <linux/slab.h>
> #include <linux/types.h>
> #include <linux/fiemap.h>
> +#include <linux/delay.h>
>
> #include <cluster/masklog.h>
>
> @@ -961,6 +962,8 @@ int ocfs2_read_virt_blocks(struct inode *inode, u64 v_block, int nr,
> int rc = 0;
> u64 p_block, p_count;
> int i, count, done = 0;
> + int retries, max_retries = 5;
> + int retry_delay_ms = 30;
>
> trace_ocfs2_read_virt_blocks(
> inode, (unsigned long long)v_block, nr, bhs, flags,
> @@ -973,7 +976,18 @@ int ocfs2_read_virt_blocks(struct inode *inode, u64 v_block, int nr,
> }
>
> while (done < nr) {
> - down_read(&OCFS2_I(inode)->ip_alloc_sem);
> + retries = 0;
> + while (retries < max_retries) {
> + if (down_read_trylock(&OCFS2_I(inode)->ip_alloc_sem))
> + break; // Lock acquired
> + msleep(retry_delay_ms);
> + retries++;
> + }

I'd like just use down_read_trylock() and fail directly if can't.

Thanks,
Joseph

> + if (retries == max_retries) {
> + rc = -EAGAIN;
> + mlog(ML_ERROR, "Cannot acquire lock\n");
> + break;
> + }
> rc = ocfs2_extent_map_get_blocks(inode, v_block + done,
> &p_block, &p_count, NULL);
> up_read(&OCFS2_I(inode)->ip_alloc_sem);