Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module

[Aditya Garg]

Hi

Ccing Ernesto

>>> Lately, I have been thinking a lot about the lack of APFS support on
>>> Linux. I was wondering what I could do about that. APFS support is not 
>>> in-tree, but there is a proprietary module sold by Paragon software [0].
>>> Obviously, this could not be used in-tree. However, there is also an 
>>> open source driver that, from what I can tell, was once planned to be 
>>> upstreamed [1] with associated filesystem progs [2]. I think I would 
>>> base most of my work off of the existing FOSS tree.
>>> 
>>> The biggest barrier I see currently is the driver's use of bufferheads.
>>> I realize that there has been a lot of work to move existing filesystem
>>> implementations to iomap/folios, and adding a filesystem that uses
>>> bufferheads would be antithetical to the purpose of that effort.
>>> Additionally, there is a lot of ifndefs/C preprocessor magic littered
>>> throughout the codebase that fixes functionality with various different
>>> versions of Linux. 
>>> 
>>> The first step would be to move away from bufferheads and the
>>> versioning. I plan to start my work in the next few weeks, and hope to
>>> have a working driver to submit to staging by the end of June. From
>>> there, I will work to have it meet more kernel standards and hopefully
>>> move into fs/ by the end of the year.
>>> 
>>> Before I started, I was wondering if anyone had any thoughts. I am open
>>> to feedback. If you think this is a bad idea, let me know. I am very
>>> passionate about the Asahi Linux project. I think this would be a good
>>> way to indirectly give back and contribute to the project. While I
>>> recognize that it is not one of Asahi's project goals (those being
>>> mostly hardware support), I am confident many users would find it
>>> helpful. I sure would.

This driver tbh will not ‘really’ be helpful as far as T2 Macs are concerned.

On these Macs, the T2 Security Chip encrypts all the APFS partitions on the internal SSD,
and the key is in the T2 Chip. Even proprietary APFS drivers cannot read these partitions.
I dunno how it works in Apple Silicon Macs.

One practical use that we actually do have with this driver is that the recovery partition of
macOS is still unencrypted. Now brcmfmac driver on Linux needs wifi firmware to work which
on these is obtained from macOS. We mount the macOS Recovery partition and use a helper
script [1] to get the firmware.

[1] https://wiki.t2linux.org/guides/wifi-bluetooth/#setting-up

>> 
>> Agreed, I think it would be helpful for many people (especially those
>> who still regularly dual boot between macOS and Linux) to have a working
>> APFS driver upstream.
>> This may also be useful once we fully bring up the Secure Enclave and need
>> to read/write to at least a single file on the xART partition which has
>> to be APFS because it's shared between all operating systems running on
>> a single machine.
>> 
>> 
>> It looks like there's still recent activity on that linux-apfs github
>> repository. Have you reached out to the people working on it to see
>> what their plans for upstreaming and/or future work are?
> 
> I did ask the upstream maintainer and he said he did not see it
> happening. He specifically mentioned the use of bufferheads as a barrier
> to mainline merging, but I get the sense that he does not have the
> time/desire to commit to upstreaming it. [0]
> 
> I did have a question/concern over the inclusion of the LZFSE/LZVN [1]
> compression library included in the module. It is BSD3 licensed, which,
> as far as I know is GPL-compatible, but what is the kernel's policy on
> external algorithms being included? It was originally developed by Apple
> and as far as I can tell is pretty necessary to read (and write)
> compressed files on APFS. Also, the library does produce an objtool
> warning.

The original author was clear about this. Its is compatible with GPL:

https://github.com/linux-apfs/linux-apfs-rw/issues/68#issuecomment-2156731178
> 
> Ted, looping you in here, your thoughts?