Re: [GIT PULL] bcachefs fixes for 6.12-rc2

[Daniel Gomez]

On Wed Oct 9, 2024 at 5:51 AM CEST, Theodore Ts'o wrote:
> On Sun, Oct 06, 2024 at 12:33:51AM -0400, Kent Overstreet wrote:
>> 
>> Correct me if I'm wrong, but your system isn't available to the
>> community, and I haven't seen a CI or dashboard for kdevops?
>
> It's up on github for anyone to download, and I've provided pre-built
> test appliance so people don't have to have downloaded xfstests and
> all of its dependencies and build it from scratch.  (That's been
> automated, of course, but the build infrastructure is setup to use a
> Debian build chroot, and with the precompiled test appliances, you can
> use my test runner on pretty much any Linux distribution; it will even
> work on MacOS if you have qemu built from macports, although for now
> you have to build the kernel on Linux distro using Parallels VM[1].)
>
> I'll note that IMHO making testing resources available to the
> community isn't really the bottleneck.  Using cloud resources,
> especially if you spin up the VM's only when you need to run the
> tests, and shut them down once the test is complete, which
> gce-xfstests does, is actually quite cheap.  At retail prices, running
> a dozen ext4 file system configurations against xfstests's "auto"
> group will take about 24 hours of VM time, and including the cost of
> the block devices, costs just under two dollars USD.  Because the
> tests are run in parallel, the total wall clock time to run all of the
> tests is about two and a half hours.  Running the "quick" group on a
> single file system configuration costs pennies.  So the $300 of free
> GCE credits will actually get someone pretty far!
>
> No, the bottleneck is having someone knowledgeable enough to interpret
> the test results and then finding the root cause of the failures.
> This is one of the reasons why I haven't stressed all that much about
> dashboards.  Dashboards are only useful if the right person(s) is
> looking at them.  That's why I've been much more interested in making
> it stupidly easy to run tests on someone's local resources, e.g.:
>
>      https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md
>
> In fact, for most people, the entry point that I envision as being
> most interesting is that they download the kvm-xfstests, and following
> the instructions in the quickstart, so they can run "kvm-xfstests
> smoke" before sending me an ext4 patch.  Running the smoke test only
> takes 15 minutes using qemu, and it's much more convenient for them to
> run that on their local machine than to trigger the test on some
> remote machine, whether it's in the cloud or someone's remote test
> server.
>
> In any case, that's why I haven't been interesting in working with
> your test infrastructure; I have my own, and in my opinion, my
> approach is the better one to make available to the community, and so
> when I have time to improve it, I'd much rather work on
> {kvm,gce,android}-xfstests.
>
> Cheers,
>
> 						- Ted
>
>
> [1] Figuring out how to coerce the MacOS toolchain to build the Linux
> kernel would be cool if anyone ever figures it out.  However, I *have*

Building Linux for arm64 is now supported in macOS. You can find all patch
series discussions here [1]. In case you want to give this a try, here the
steps:

	```shell
	diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux
	```
	
	```shell
	brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config
	```
	
	```shell
	brew tap bee-headers/bee-headers
	brew install bee-headers/bee-headers/bee-headers
	```
	
	Initialize the environment with `bee-init`. Repeat with every new shell:
	
	```shell
	source bee-init
	```
	
	```shell
	make LLVM=1 defconfig
	make LLVM=1 -j$(nproc)
	```
	
More details about the setup required can be found here [2].

This allows to build the kernel and boot it with QEMU -kernel argument. And
debug it with with lldb.

[1]
v3: https://lore.kernel.org/all/20240925-macos-build-support-v3-1-233dda880e60@xxxxxxxxxxx/
v2: https://lore.kernel.org/all/20240906-macos-build-support-v2-0-06beff418848@xxxxxxxxxxx/
v1: https://lore.kernel.org/all/20240807-macos-build-support-v1-0-4cd1ded85694@xxxxxxxxxxx/

[2] https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md

Daniel

> done kernel development using a Macbook Air M2 while on a cruise ship
> with limited internet access, building the kernel using a Parallels VM
> running Debian testing, and then using qemu from MacPorts to avoid the
> double virtualization performance penalty to run xfstests to test the
> freshly-built arm64 kernel, using my xfstests runner -- and all of
> this is available on github for anyone to use.