Re: [PATCH 0/5] mm: rework hmm to use devm_memremap_pages

[Dan Williams]

On Tue, Jun 5, 2018 at 3:19 PM, Dave Airlie <airlied@xxxxxxxxx> wrote:
> On 6 June 2018 at 04:48, Jerome Glisse <jglisse@xxxxxxxxxx> wrote:
>> On Tue, May 29, 2018 at 04:33:49PM -0700, Dan Williams wrote:
>>> On Tue, May 29, 2018 at 4:00 PM, Dave Airlie <airlied@xxxxxxxxx> wrote:
>>> > On 30 May 2018 at 08:31, Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
>>> >> On Tue, May 29, 2018 at 3:22 PM, Dave Airlie <airlied@xxxxxxxxx> wrote:
>>> >>>
>>> >>> On 24 May 2018 at 13:18, Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
>>> >>> > On Wed, May 23, 2018 at 5:10 PM, Jerome Glisse <jglisse@xxxxxxxxxx> wrote:
>>> >>> >> On Mon, May 21, 2018 at 03:35:14PM -0700, Dan Williams wrote:
>>> >>> >>> Hi Andrew, please consider this series for 4.18.
>>> >>> >>>
>>> >>> >>> For maintainability, as ZONE_DEVICE continues to attract new users,
>>> >>> >>> it is useful to keep all users consolidated on devm_memremap_pages() as
>>> >>> >>> the interface for create "device pages".
>>> >>> >>>
>>> >>> >>> The devm_memremap_pages() implementation was recently reworked to make
>>> >>> >>> it more generic for arbitrary users, like the proposed peer-to-peer
>>> >>> >>> PCI-E enabling. HMM pre-dated this rework and opted to duplicate
>>> >>> >>> devm_memremap_pages() as hmm_devmem_pages_create().
>>> >>> >>>
>>> >>> >>> Rework HMM to be a consumer of devm_memremap_pages() directly and fix up
>>> >>> >>> the licensing on the exports given the deep dependencies on the mm.
>>> >>> >>
>>> >>> >> I am on PTO right now so i won't be able to quickly review it all
>>> >>> >> but forcing GPL export is problematic for me now. I rather have
>>> >>> >> device driver using "sane" common helpers than creating their own
>>> >>> >> crazy thing.
>>> >>> >
>>> >>> > Sane drivers that need this level of deep integration with Linux
>>> >>> > memory management need to be upstream. Otherwise, HMM is an
>>> >>> > unprecedented departure from the norms of Linux kernel development.
>>> >>>
>>> >>> Isn't it the author of code choice what EXPORT_SYMBOL to use? and
>>> >>> isn't the agreement that if something is EXPORT_SYMBOL now, changing
>>> >>> underlying exports isn't considered a good idea. We've seen this before
>>> >>> with the refcount fun,
>>> >>>
>>> >>> See d557d1b58b3546bab2c5bc2d624c5709840e6b10
>>> >>>
>>> >>> Not commenting on the legality or what derived works are considered,
>>> >>> since really the markings are just an indication of the authors opinion,
>>> >>> and at this stage I think are actually meaningless, since we've diverged
>>> >>> considerably from the advice given to Linus back when this started.
>>> >>
>>> >> Yes, and in this case devm_memremap_pages() was originally written by
>>> >> Christoph and I:
>>> >>
>>> >>     41e94a851304 add devm_memremap_pages
>>> >
>>> > So you wrote some code in 2015 (3 years ago) and you've now decided
>>> > to change the EXPORT marker on it? what changed in 3 years, and why
>>> > would changing that marker 3 years later have any effect on your original
>>> > statement that it was an EXPORT_SYMBOL.
>>> >
>>> > Think what EXPORT_SYMBOL vs GPL means, it isn't a bit stick that magically
>>> > makes things into derived works. If something wasn't a derived work for 3 years
>>> > using that API, then it isn't a derived work now 3 years later because you
>>> > changed the marker. Retrospectively changing the markers doesn't really
>>> > make any sense legally or otherwise.
>>>
>>> It honestly was an oversight, and as we've gone on to add deeper and
>>> deeper ties into the mm and filesystems [1] I realized this symbol was
>>> mis-labeled.  It would be one thing if this was just some random
>>> kernel leaf / library function, but this capability when turned on
>>> causes the entire kernel to be recompiled as things like the
>>> definition of put_page() changes. It's deeply integrated with how
>>> Linux manages memory.
>>
>> I am personaly on the fence on deciding GPL versus non GPL export
>> base on subjective view of what is deeply integrated and what is
>> not. I think one can argue that every single linux kernel function
>> is deeply integrated within the kernel, starting with all device
>> drivers functions. One could similarly argue that nothing is ...
>
> This is the point I wasn't making so well, the whole deciding on a derived
> work from the pov of one of the works isn't really going to be how a court
> looks at it.
>
> At day 0, you have a Linux kernel, and a separate Windows kernel driver,
> clearly they are not derived works.
>
> You add interfaces to the Windows kernel driver and it becomes a Linux
> kernel driver, you never ship them together, derived work only if those
> interfaces are GPL only? or derived work only if shipped together?
> only shipped together and GPL only? Clearly not a clearcut case here.
>
> The code base is 99% the same, the kernel changes an export to a GPL
> export, the external driver hasn't changed one line of code, and it suddenly
> becomes a derived work?
>
> Oversights happen, but 3 years of advertising an interface under the non-GPL
> and changing it doesn't change whether the external driver is derived or not,
> nor will it change anyone's legal position.

My concern is the long term health and maintainability of the Linux
kernel. HMM exports deep Linux internals out to proprietary drivers
with no way for folks in the wider kernel community to validate that
the interfaces are necessary or sufficient besides "take Jerome's word
for it". Every time I've pushed back on any HMM feature the response
is something to the effect of, "no, out of tree drivers need this".
HMM needs to grow upstream users and the functionality needs to be
limited to whatever those upstream users exploit. Since there are no
upstream users of HMM, we should delete it unless / until those users
arrive.

I want the EXPORT_SYMBOL_GPL on devm_memremap_pages() primarily for
development purposes. Any new users of devm_memremap_pages() should be
aware that they are subscribing to the whims of the core-VM, i.e. the
ongoing evolution of 'struct page', and encourage those drivers to be
upstream to improve the implementation, and consolidate use cases. I'm
not qualified to comment on your "nor will it change anyone's legal
position.", but I'm saying it's in the Linux kernel's best interest
that new users of this interface assume they need to be GPL.