[GIT PULL] bcachefs

[Kent Overstreet]

Hi Linus,

Here it is, the bcachefs pull request. For brevity the list of patches
below is only the initial part of the series, the non-bcachefs prep
patches and the first bcachefs patch, but the diffstat is for the entire
series.

Six locks has all the changes you suggested, text size went down
significantly. If you'd still like this to see more review from the
locking people, I'm not against them living in fs/bcachefs/ as an
interim; perhaps Dave could move them back to kernel/locking when he
starts using them or when locking people have had time to look at them -
I'm just hoping for this to not block the merge.

Recently some people have expressed concerns about "not wanting a repeat
of ntfs3" - from what I understand the issue there was just severe
buggyness, so perhaps showing the bcachefs automated test results will
help with that:

  https://evilpiepirate.org/~testdashboard/ci

The main bcachefs branch runs fstests and my own test suite in several
varations, including lockdep+kasan, preempt, and gcov (we're at 82% line
coverage); I'm not currently seeing any lockdep or kasan splats (or
panics/oopses, for that matter).

(Worth noting the bug causing the most test failures by a wide margin is
actually an io_uring bug that causes random umount failures in shutdown
tests. Would be great to get that looked at, it doesn't just affect
bcachefs).

Regarding feature status - most features are considered stable and ready
for use, snapshots and erasure coding are both nearly there. But a
filesystem on this scale is a massive project, adequately conveying the
status of every feature would take at least a page or two.

We may want to mark it as EXPERIMENTAL for a few releases, I haven't
done that as yet. (I wouldn't consider single device without snapshots
to be experimental, but - given that the number of users and bug reports
is about to shoot up, perhaps I should...).

Cheers,
Kent

---------

The following changes since commit 6995e2de6891c724bfeb2db33d7b87775f913ad1:

  Linux 6.4 (2023-06-25 16:29:58 -0700)

are available in the Git repository at:

  https://evilpiepirate.org/git/bcachefs.git bcachefs-for-upstream

for you to fetch changes up to 66012992c01af99f07ac696a8c9563ba291c1e7f

  bcachefs: fsck needs BTREE_UPDATE_INTERNAL_SNAPSHOT_NODE

----------------------------------------------------------------
Christopher James Halse Rogers (1):
      stacktrace: Export stack_trace_save_tsk

Daniel Hill (1):
      lib: add mean and variance module.

Kent Overstreet (26):
      Compiler Attributes: add __flatten
      locking/lockdep: lock_class_is_held()
      locking/lockdep: lockdep_set_no_check_recursion()
      locking: SIX locks (shared/intent/exclusive)
      MAINTAINERS: Add entry for six locks
      sched: Add task_struct->faults_disabled_mapping
      fs: factor out d_mark_tmpfile()
      block: Add some exports for bcachefs
      block: Allow bio_iov_iter_get_pages() with bio->bi_bdev unset
      block: Bring back zero_fill_bio_iter
      block: Don't block on s_umount from __invalidate_super()
      bcache: move closures to lib/
      MAINTAINERS: Add entry for closures
      closures: closure_wait_event()
      closures: closure_nr_remaining()
      closures: Add a missing include
      iov_iter: copy_folio_from_iter_atomic()
      MAINTAINERS: Add entry for generic-radix-tree
      lib/generic-radix-tree.c: Don't overflow in peek()
      lib/generic-radix-tree.c: Add a missing include
      lib/generic-radix-tree.c: Add peek_prev()
      lib/string_helpers: string_get_size() now returns characters wrote
      lib: Export errname
      mean_and_variance: Assorted fixes/cleanups
      MAINTAINERS: Add entry for bcachefs
      bcachefs: Initial commit

 MAINTAINERS                                     |   39 +
 block/bdev.c                                    |    2 +-
 block/bio.c                                     |   18 +-
 block/blk-core.c                                |    1 +
 block/blk.h                                     |    1 -
 drivers/md/bcache/Kconfig                       |   10 +-
 drivers/md/bcache/Makefile                      |    4 +-
 drivers/md/bcache/bcache.h                      |    2 +-
 drivers/md/bcache/super.c                       |    1 -
 drivers/md/bcache/util.h                        |    3 +-
 fs/Kconfig                                      |    1 +
 fs/Makefile                                     |    1 +
 fs/bcachefs/Kconfig                             |   75 +
 fs/bcachefs/Makefile                            |   74 +
 fs/bcachefs/acl.c                               |  414 +++
 fs/bcachefs/acl.h                               |   58 +
 fs/bcachefs/alloc_background.c                  | 2212 +++++++++++++
 fs/bcachefs/alloc_background.h                  |  251 ++
 fs/bcachefs/alloc_foreground.c                  | 1534 +++++++++
 fs/bcachefs/alloc_foreground.h                  |  224 ++
 fs/bcachefs/alloc_types.h                       |  122 +
 fs/bcachefs/backpointers.c                      |  884 +++++
 fs/bcachefs/backpointers.h                      |  131 +
 fs/bcachefs/bbpos.h                             |   48 +
 fs/bcachefs/bcachefs.h                          | 1139 +++++++
 fs/bcachefs/bcachefs_format.h                   | 2219 +++++++++++++
 fs/bcachefs/bcachefs_ioctl.h                    |  368 +++
 fs/bcachefs/bkey.c                              | 1063 ++++++
 fs/bcachefs/bkey.h                              |  774 +++++
 fs/bcachefs/bkey_buf.h                          |   61 +
 fs/bcachefs/bkey_cmp.h                          |  129 +
 fs/bcachefs/bkey_methods.c                      |  520 +++
 fs/bcachefs/bkey_methods.h                      |  169 +
 fs/bcachefs/bkey_sort.c                         |  201 ++
 fs/bcachefs/bkey_sort.h                         |   44 +
 fs/bcachefs/bset.c                              | 1588 +++++++++
 fs/bcachefs/bset.h                              |  541 ++++
 fs/bcachefs/btree_cache.c                       | 1213 +++++++
 fs/bcachefs/btree_cache.h                       |  106 +
 fs/bcachefs/btree_gc.c                          | 2130 ++++++++++++
 fs/bcachefs/btree_gc.h                          |  112 +
 fs/bcachefs/btree_io.c                          | 2261 +++++++++++++
 fs/bcachefs/btree_io.h                          |  228 ++
 fs/bcachefs/btree_iter.c                        | 3214 ++++++++++++++++++
 fs/bcachefs/btree_iter.h                        |  916 ++++++
 fs/bcachefs/btree_key_cache.c                   | 1075 ++++++
 fs/bcachefs/btree_key_cache.h                   |   48 +
 fs/bcachefs/btree_locking.c                     |  804 +++++
 fs/bcachefs/btree_locking.h                     |  424 +++
 fs/bcachefs/btree_types.h                       |  731 +++++
 fs/bcachefs/btree_update.h                      |  358 ++
 fs/bcachefs/btree_update_interior.c             | 2476 ++++++++++++++
 fs/bcachefs/btree_update_interior.h             |  327 ++
 fs/bcachefs/btree_update_leaf.c                 | 2050 ++++++++++++
 fs/bcachefs/btree_write_buffer.c                |  343 ++
 fs/bcachefs/btree_write_buffer.h                |   14 +
 fs/bcachefs/btree_write_buffer_types.h          |   44 +
 fs/bcachefs/buckets.c                           | 2200 +++++++++++++
 fs/bcachefs/buckets.h                           |  370 +++
 fs/bcachefs/buckets_types.h                     |   92 +
 fs/bcachefs/buckets_waiting_for_journal.c       |  166 +
 fs/bcachefs/buckets_waiting_for_journal.h       |   15 +
 fs/bcachefs/buckets_waiting_for_journal_types.h |   23 +
 fs/bcachefs/chardev.c                           |  769 +++++
 fs/bcachefs/chardev.h                           |   31 +
 fs/bcachefs/checksum.c                          |  712 ++++
 fs/bcachefs/checksum.h                          |  215 ++
 fs/bcachefs/clock.c                             |  193 ++
 fs/bcachefs/clock.h                             |   38 +
 fs/bcachefs/clock_types.h                       |   37 +
 fs/bcachefs/compress.c                          |  638 ++++
 fs/bcachefs/compress.h                          |   18 +
 fs/bcachefs/counters.c                          |  107 +
 fs/bcachefs/counters.h                          |   17 +
 fs/bcachefs/darray.h                            |   87 +
 fs/bcachefs/data_update.c                       |  565 ++++
 fs/bcachefs/data_update.h                       |   43 +
 fs/bcachefs/debug.c                             |  957 ++++++
 fs/bcachefs/debug.h                             |   32 +
 fs/bcachefs/dirent.c                            |  564 ++++
 fs/bcachefs/dirent.h                            |   68 +
 fs/bcachefs/disk_groups.c                       |  548 ++++
 fs/bcachefs/disk_groups.h                       |  101 +
 fs/bcachefs/ec.c                                | 1957 +++++++++++
 fs/bcachefs/ec.h                                |  261 ++
 fs/bcachefs/ec_types.h                          |   41 +
 fs/bcachefs/errcode.c                           |   63 +
 fs/bcachefs/errcode.h                           |  243 ++
 fs/bcachefs/error.c                             |  297 ++
 fs/bcachefs/error.h                             |  213 ++
 fs/bcachefs/extent_update.c                     |  173 +
 fs/bcachefs/extent_update.h                     |   12 +
 fs/bcachefs/extents.c                           | 1384 ++++++++
 fs/bcachefs/extents.h                           |  755 +++++
 fs/bcachefs/extents_types.h                     |   40 +
 fs/bcachefs/eytzinger.h                         |  281 ++
 fs/bcachefs/fifo.h                              |  127 +
 fs/bcachefs/fs-common.c                         |  501 +++
 fs/bcachefs/fs-common.h                         |   43 +
 fs/bcachefs/fs-io.c                             | 3948 +++++++++++++++++++++++
 fs/bcachefs/fs-io.h                             |   54 +
 fs/bcachefs/fs-ioctl.c                          |  556 ++++
 fs/bcachefs/fs-ioctl.h                          |   81 +
 fs/bcachefs/fs.c                                | 1943 +++++++++++
 fs/bcachefs/fs.h                                |  206 ++
 fs/bcachefs/fsck.c                              | 2494 ++++++++++++++
 fs/bcachefs/fsck.h                              |    8 +
 fs/bcachefs/inode.c                             |  868 +++++
 fs/bcachefs/inode.h                             |  192 ++
 fs/bcachefs/io.c                                | 3056 ++++++++++++++++++
 fs/bcachefs/io.h                                |  202 ++
 fs/bcachefs/io_types.h                          |  165 +
 fs/bcachefs/journal.c                           | 1453 +++++++++
 fs/bcachefs/journal.h                           |  520 +++
 fs/bcachefs/journal_io.c                        | 1868 +++++++++++
 fs/bcachefs/journal_io.h                        |   64 +
 fs/bcachefs/journal_reclaim.c                   |  863 +++++
 fs/bcachefs/journal_reclaim.h                   |   86 +
 fs/bcachefs/journal_sb.c                        |  219 ++
 fs/bcachefs/journal_sb.h                        |   24 +
 fs/bcachefs/journal_seq_blacklist.c             |  322 ++
 fs/bcachefs/journal_seq_blacklist.h             |   22 +
 fs/bcachefs/journal_types.h                     |  358 ++
 fs/bcachefs/keylist.c                           |   52 +
 fs/bcachefs/keylist.h                           |   74 +
 fs/bcachefs/keylist_types.h                     |   16 +
 fs/bcachefs/lru.c                               |  178 +
 fs/bcachefs/lru.h                               |   63 +
 fs/bcachefs/migrate.c                           |  182 ++
 fs/bcachefs/migrate.h                           |    7 +
 fs/bcachefs/move.c                              | 1162 +++++++
 fs/bcachefs/move.h                              |   96 +
 fs/bcachefs/move_types.h                        |   36 +
 fs/bcachefs/movinggc.c                          |  420 +++
 fs/bcachefs/movinggc.h                          |   12 +
 fs/bcachefs/nocow_locking.c                     |  123 +
 fs/bcachefs/nocow_locking.h                     |   49 +
 fs/bcachefs/nocow_locking_types.h               |   20 +
 fs/bcachefs/opts.c                              |  555 ++++
 fs/bcachefs/opts.h                              |  543 ++++
 fs/bcachefs/printbuf.c                          |  415 +++
 fs/bcachefs/printbuf.h                          |  284 ++
 fs/bcachefs/quota.c                             |  980 ++++++
 fs/bcachefs/quota.h                             |   72 +
 fs/bcachefs/quota_types.h                       |   43 +
 fs/bcachefs/rebalance.c                         |  363 +++
 fs/bcachefs/rebalance.h                         |   28 +
 fs/bcachefs/rebalance_types.h                   |   26 +
 fs/bcachefs/recovery.c                          | 1648 ++++++++++
 fs/bcachefs/recovery.h                          |   58 +
 fs/bcachefs/reflink.c                           |  388 +++
 fs/bcachefs/reflink.h                           |   79 +
 fs/bcachefs/replicas.c                          | 1056 ++++++
 fs/bcachefs/replicas.h                          |   91 +
 fs/bcachefs/replicas_types.h                    |   27 +
 fs/bcachefs/seqmutex.h                          |   48 +
 fs/bcachefs/siphash.c                           |  173 +
 fs/bcachefs/siphash.h                           |   87 +
 fs/bcachefs/str_hash.h                          |  370 +++
 fs/bcachefs/subvolume.c                         | 1505 +++++++++
 fs/bcachefs/subvolume.h                         |  167 +
 fs/bcachefs/subvolume_types.h                   |   22 +
 fs/bcachefs/super-io.c                          | 1597 +++++++++
 fs/bcachefs/super-io.h                          |  126 +
 fs/bcachefs/super.c                             | 1993 ++++++++++++
 fs/bcachefs/super.h                             |  266 ++
 fs/bcachefs/super_types.h                       |   51 +
 fs/bcachefs/sysfs.c                             | 1064 ++++++
 fs/bcachefs/sysfs.h                             |   48 +
 fs/bcachefs/tests.c                             |  939 ++++++
 fs/bcachefs/tests.h                             |   15 +
 fs/bcachefs/trace.c                             |   16 +
 fs/bcachefs/trace.h                             | 1247 +++++++
 fs/bcachefs/two_state_shared_lock.c             |    8 +
 fs/bcachefs/two_state_shared_lock.h             |   59 +
 fs/bcachefs/util.c                              | 1137 +++++++
 fs/bcachefs/util.h                              |  842 +++++
 fs/bcachefs/varint.c                            |  121 +
 fs/bcachefs/varint.h                            |   11 +
 fs/bcachefs/vstructs.h                          |   63 +
 fs/bcachefs/xattr.c                             |  648 ++++
 fs/bcachefs/xattr.h                             |   51 +
 fs/dcache.c                                     |   12 +-
 fs/super.c                                      |   40 +-
 include/linux/bio.h                             |    7 +-
 include/linux/blkdev.h                          |    1 +
 {drivers/md/bcache => include/linux}/closure.h  |   46 +-
 include/linux/compiler_attributes.h             |    5 +
 include/linux/dcache.h                          |    1 +
 include/linux/exportfs.h                        |    6 +
 include/linux/fs.h                              |    1 +
 include/linux/generic-radix-tree.h              |   68 +-
 include/linux/lockdep.h                         |   10 +
 include/linux/lockdep_types.h                   |    2 +-
 include/linux/mean_and_variance.h               |  198 ++
 include/linux/sched.h                           |    1 +
 include/linux/six.h                             |  388 +++
 include/linux/string_helpers.h                  |    4 +-
 include/linux/uio.h                             |    2 +
 init/init_task.c                                |    1 +
 kernel/Kconfig.locks                            |    3 +
 kernel/locking/Makefile                         |    1 +
 kernel/locking/lockdep.c                        |   46 +
 kernel/locking/six.c                            |  893 +++++
 kernel/stacktrace.c                             |    2 +
 lib/Kconfig                                     |    3 +
 lib/Kconfig.debug                               |   18 +
 lib/Makefile                                    |    2 +
 {drivers/md/bcache => lib}/closure.c            |   36 +-
 lib/errname.c                                   |    1 +
 lib/generic-radix-tree.c                        |   76 +-
 lib/iov_iter.c                                  |   53 +-
 lib/math/Kconfig                                |    3 +
 lib/math/Makefile                               |    2 +
 lib/math/mean_and_variance.c                    |  158 +
 lib/math/mean_and_variance_test.c               |  239 ++
 lib/string_helpers.c                            |    8 +-
 217 files changed, 92440 insertions(+), 86 deletions(-)