Re: [PATCH Vx 1/1] f2fs: Avoid deadlock between writeback and checkpoint

[Chao Yu]

On 2021/11/18 0:56, Jaegeuk Kim wrote:
> On 11/09, niuzhiguo84@xxxxxxxxx wrote:
> > From: Zhiguo Niu <zhiguo.niu@xxxxxxxxxx>
> >
> > There could be a scenario as following:
> > The inodeA and inodeB are in b_io queue of writeback
> > inodeA : f2fs's node inode
> > inodeB : a dir inode with only one dirty pages, and the node page
> > of inodeB cached into inodeA
> >
> > writeback:
> >
> > wb_workfn
> > wb_writeback
> > blk_start_plug
> >          loop {
> >          queue_io
> >          progress=__writeback_inodes_wb
> >                  __writeback_single_inode
> >                          do_writepages
> >                                  f2fs_write_data_pages
> >                                  wbc->pages_skipped +=get_dirty_pages
> >                          inode->i_state &= ~dirty
> >                  wrote++
> >                  requeue_inode
> >          }
> > blk_finish_plug
> >
> > checkpoint:
> >
> > f2fs_write_checkpoint
> > f2fs_sync_dirty_inodes
> > filemap_fdatawrite
> > do_writepages
> > f2fs_write_data_pages
> >          f2fs_write_single_data_page
> >                  f2fs_do_write_data_page
> >                          set_page_writeback
> >                          f2fs_outplace_write_data
> >                                  f2fs_update_data_blkaddr
> >                                          f2fs_wait_on_page_writeback
> >                  inode_dec_dirty_pages
> >
> > 1. Writeback thread flush inodeA, and push it's bio request in task's plug;
> > 2. Checkpoint thread writes inodeB's dirty page, and then wait its node
> >      page writeback cached into inodeA which is in writeback task's plug
> > 3. Writeback thread flush inodeB and skip writing the dirty page as
> >      wb_sync_req[DATA] > 0.
> > 4. As none of the inodeB's page is marked as PAGECACHE_TAG_DIRTY, writeback
> >      thread clear inodeB's dirty state.
> > 5. Then inodeB is moved from b_io to b_dirty because of pages_skipped > 0
> >      as checkpoint thread is stuck before dec dirty_pages.
> >
> > This patch collect correct pages_skipped according to the tag state in
> > page tree of inode
> >
> > Signed-off-by: Zhiguo Niu <zhiguo.niu@xxxxxxxxxx>
> > Signed-off-by: Jing Xia <jing.xia@xxxxxxxxxx>
> > ---
> >   fs/f2fs/data.c | 4 +++-
> >   1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> > index f4fd6c246c9a..e98628e3868c 100644
> > --- a/fs/f2fs/data.c
> > +++ b/fs/f2fs/data.c
> > @@ -3237,7 +3237,9 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
> >   	return ret;
> >   
> >   skip_write:
> > -	wbc->pages_skipped += get_dirty_pages(inode);
> > +	wbc->pages_skipped +=
> > +		mapping_tagged(inode->i_mapping, PAGECACHE_TAG_DIRTY) ?
>
> Is there any race condition to get 0, if there's any dirty page? IOWs, it

Quoted from Jing Xia's explanation:

[T:writeback]				[T:checkpoint]
					- do_writepages  -- sync write inodeB, inc wb_sync_req[DATA]
					 - f2fs_write_data_pages
					  - f2fs_write_single_data_page -- write last dirty page
					   - f2fs_do_write_data_page
					    - set_page_writeback  -- clear page dirty flag and
					    PAGECACHE_TAG_DIRTY tag in radix tree
					    - f2fs_outplace_write_data
					     - f2fs_update_data_blkaddr
					      - f2fs_wait_on_page_writeback -- wait NodeA to writeback here
					   - inode_dec_dirty_pages
bio contains NodeA was plugged in writeback threads
- writeback_sb_inodes
 - writeback_single_inode
  - do_writepages
   - f2fs_write_data_pages -- skip writepages due to wb_sync_req[DATA]
    - wbc->pages_skipped += get_dirty_pages() -- PAGECACHE_TAG_DIRTY is not set but get_dirty_pages() returns one
 - requeue_inode -- requeue inode to wb->b_dirty queue due to non-zero.pages_skipped

> seems the current condition is just requeuing the inode as dirty, but next
> flushing time will remove it from dirty list. Is this giving too much overheads?

I prefer to let writeback thread call blk_flush_plug() after skipping
writepages() due to wb_sync_req[DATA/NODE] check condition, thoughts?

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 9f754aaef558..b6e1ed73f8f5 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -3087,6 +3087,8 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
 			/* give a priority to WB_SYNC threads */
 			if (atomic_read(&sbi->wb_sync_req[DATA]) &&
 					wbc->sync_mode == WB_SYNC_NONE) {
+				if (current->plug)
+					blk_flush_plug(current->plug, false);
 				done = 1;
 				break;
 			}
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 556fcd8457f3..dd9a817d8dab 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -1946,6 +1946,8 @@ int f2fs_sync_node_pages(struct f2fs_sb_info *sbi,
 			if (atomic_read(&sbi->wb_sync_req[NODE]) &&
 					wbc->sync_mode == WB_SYNC_NONE) {
 				done = 1;
+				if (current->plug)
+					blk_flush_plug(current->plug, false);
 				break;
 			}



Thanks,

> > +		get_dirty_pages(inode) : 0;
> >   	trace_f2fs_writepages(mapping->host, wbc, DATA);
> >   	return 0;
> >   }
> > --
> > 2.28.0