Re: [GIT PULL] Btrfs updates for 6.2 (updated merge log)

[David Sterba]

On Tue, Dec 13, 2022 at 08:09:29AM +0800, Qu Wenruo wrote:
> > - raid56 reliability vs performance trade off
> >    - fix destructive RMW for raid5 data (raid6 still needs work) - do full RMW
> >      cycle for writes and verify all checksums before overwrite, this should
> >      prevent rewriting potentially corrupted data without notice
> 
> Unfortunately, the "RMW" term seems abused.

I used is as a shortcut but it's probably confusing, thanks for the
suggested updates.

> >    - stripes are cached in memory which should reduce the performance impact but
> >      still can hurt some workloads
> 
> The cache behavior is not changed in this big chunk of raid56 work, but 
> commit f6065f8edeb2 ("btrfs: raid56: don't trust any cached sector in 
> __raid56_parity_recover()") is still the main thing affecting recovery path.
> 
> Thus although we didn't change the cache policy, it will still be bad 
> for recovery cases (missing device, or some sector has mimsatch csum).

Yeah, there's no change but I felt it should be mentioned together with
the RMW as it'll be used more than before.

Linus, below is the complete merge log with the edits.

---
User visible features:

- raid56 reliability vs performance trade off
  - fix destructive RMW for raid5 data (raid6 still needs work) - do full
    checksum verification for all data during RMW cycle, this should prevent
    rewriting potentially corrupted data without notice
  - stripes are cached in memory which should reduce the performance impact but
    still can hurt some workloads
  - checksums are verified after repair again
  - this is the last option without introducing additional features (write
    intent bitmap, journal, another tree), the extra checksum read/verification
    was supposed to be avoided by the original implementation exactly for
    performance reasons but that caused all the reliability problems

- discard=async by default for devices that support it

- implement emergency flush reserve to avoid almost all unnecessary transaction
  aborts due to ENOSPC in cases where there are too many delayed refs or
  delayed allocation

- skip block group synchronization if there's no change in used bytes, can
  reduce transaction commit count for some workloads

Performance improvements:

- fiemap and lseek
  - overall speedup due to skipping unnecessary or duplicate searches (-40% run time)
  - cache some data structures and sharedness of extents (-30% run time)

- send
  - faster backref resolution when finding clones
  - cached leaf to root mapping for faster backref walking
  - improved clone/sharing detection
  - overall run time improvements (-70%)

Core:

- module initialization converted to a table of function pointers run in a
  sequence

- preparation for fscrypt, extend passing file names across calls, dir item can
  store encryption status

- raid56 updates
  - more accurate error tracking of sectors within stripe
  - simplify recovery path and remove dedicated endio worker kthread
  - simplify scrub call paths
  - refactoring to support the extra data checksum verification during RMW
    cycle

- tree block parentness checks consolidated and done at metadata read time

- improved error handling

- cleanups
  - move a lot of code for better synchronization between kernel and user space
    sources, split big files
  - enum cleanups
  - GFP flag cleanups
  - header file cleanups, prototypes, dependencies
  - redundant parameter cleanups
  - inline extent handling simplifications
  - inode parameter conversion
  - data structure cleanups, reductions, renames, merges