Re: [GIT PULL] bcachefs fixes for 6.12-rc2

[Kent Overstreet]

On Mon, Oct 07, 2024 at 10:58:47AM GMT, Josef Bacik wrote:
> I tend to ignore these kind of emails, it's been a decade and weirdly the file
> system development community likes to use btrfs as a punching bag.  I honestly
> don't care what anybody else thinks, but I've gotten feedback from others in the
> community that they wish I'd say something when somebody says things so patently
> false.  So I'm going to respond exactly once to this, and it'll be me satisfying
> my quota for this kind of thing for the rest of the year.
> 
> Btrfs is used by default in the desktop spin of Fedora, openSuse, and maybe some
> others.  Our development community is actively plugged into those places, we
> drop everything to help when issues arise there.  Btrfs is the foundation of the
> Meta fleet.  We rely on its capabilities and, most importantly of all, its
> stability for our infrastructure.
> 
> Is it perfect?  Absolutely not.  You will never hear me say that.  I have often,
> and publicly, said that Meta also uses XFS in our database workloads, because it
> simply is just better than Btrfs at that.
> 
> Yes, XFS is better at Btrfs at some things.  I'm not afraid to admit that,
> because my personal worth is not tied to the software projects I'm involved in.
> Dave Chinner, Darrick Wong, Christoph Hellwig, Eric Sandeen, and many others have
> done a fantastic job with XFS.  I have a lot of respect for them and the work
> they've done.  I've learned a lot from them.
> 
> Ext4 is better at Btrfs in a lot of those same things.  Ted T'so, Andreas
> Dilger, Jan Kara, and many others have done a fantastic job with ext4.
> 
> I have learned a lot from all of these developers, all of these file systems,
> and many others in this community.
> 
> Bcachefs is doing new and interesting things.  There are many things that I see
> you do that I wish we had the foresight to know were going to be a problem with
> Btrfs and done it differently.  You, along with the wider file system community,
> have a lot of the same ideals, same practices, and same desire to do your
> absolute best work.  That is an admirable trait, one that we all share.
> 
> But dragging other people and their projects down is not the sort of behavior
> that I think should have a place in this community.  This is not the kind of
> community I want to exist in.  You are not the only person who does this, but
> you are the most vocal and constant example of it.  Just like I tell my kids,
> just because somebody else is doing something wrong doesn't mean you get to do
> it too.

Josef, I've got to be honest with you, if 10 years in one filesystem
still has a lot of user reports that clearly aren't being addressed
where the filesystem is wedging itself, that's a pretty epic fail and
that really is the main reason why I'm here.

#1 priority in filesystem land has to be robustness. Not features, not
performance; it has to simply work.

The bar for "acceptably good" is really, really high when you're
responsible for user's data. In the rest of the kernel, if you screw up,
generally the worst that happens is you crash the machine - users are
annoyed, whatever they were doing gets interrupted, but nothing
drastically bad happens.

In filesystem land, fairly minor screwups can lead to the entire machine
being down for extended periods of time if the filesystem has wedged
itself and really involved repair procedures that users _should not_
have to do, or worst real data loss. And you need to be thinking about
the trust that users are placing in you; that's people's _lives_ they're
storing on their machines.

So no, based on the feedback I still _regularly_ get I don't think btrfs
hit an acceptable level of reliability, and if it's taking this long I
doubt it will.

"Mostly works" is just not good enough.

To be fair, bcachefs isn't "good enough" yet either, I'm still getting
bug reports where bcachefs wedges itself too.

But I've also been pretty explicit about that, and I'm not taking the
experimental label off until those reports have stopped and we've
addressed _every_ known way it can wedge itself and we've torture tested
the absolute crap out of repair.

And I think you've set the bar too low, by just accepting that btrfs
isn't going to be as good as xfs in some situations.

I don't think there's any reason a modern COW filesystem has to be
crappier in _any_ respect than ext4/xfs. It's just a matter of
prioritizing the essentials and working at it until it's done.