Re: [PATCH 09/10] mm/hmm: allow to mirror vma of a file on a DAX backed filesystem

[Jerome Glisse]

On Wed, Jan 30, 2019 at 07:28:12PM -0800, Dan Williams wrote:
> On Wed, Jan 30, 2019 at 10:36 AM Jerome Glisse <jglisse@xxxxxxxxxx> wrote:
> [..]
> > > > This
> > > > is one of the motivation behind HMM ie have it as an impedence layer
> > > > between mm and device drivers so that mm folks do not have to under-
> > > > stand every single device driver but only have to understand the
> > > > contract HMM has with all device driver that uses it.
> > >
> > > This gets to heart of my critique of the approach taken with HMM. The
> > > above statement is antithetical to
> > > Documentation/process/stable-api-nonsense.rst. If HMM is trying to set
> > > expectations that device-driver projects can write to a "stable" HMM
> > > api then HMM is setting those device-drivers up for failure.
> >
> > So i am not expressing myself correctly. If someone want to change mm
> > in anyway that would affect HMM user, it can and it is welcome too
> > (assuming that those change are wanted by the community and motivated
> > for good reasons). Here by understanding HMM contract and preserving it
> > what i mean is that all you have to do is update the HMM API in anyway
> > that deliver the same result to the device driver. So what i means is
> > that instead of having to understand each device driver. For instance
> > you have HMM provide X so that driver can do Y; then what can be Z a
> > replacement for X that allow driver to do Y. The point here is that
> > HMM define what Y is and provide X for current kernel mm code. If X
> > ever need to change so that core mm can evolve than you can and are
> > more than welcome to do it. With HMM Y is defined and you only need to
> > figure out how to achieve the same end result for the device driver.
> >
> > The point is that you do not have to go read each device driver to
> > figure out Y.driver_foo, Y.driver_bar, ... you only have HMM that
> > define what Y means and is ie this what device driver are trying to
> > do.
> >
> > Obviously here i assume that we do not want to regress features ie
> > we want to keep device driver features intact when we modify anything.
> 
> The specific concern is HMM attempting to expand the regression
> boundary beyond drivers that exist in the kernel. The regression
> contract that has priority is the one established for in-tree users.
> If an in-tree change to mm semantics is fine for in-tree mm users, but
> breaks out of tree users the question to those out of tree users is
> "why isn't your use case upstream?". HMM is not that use case in and
> of itself.

I do not worry about out of tree user and we should not worry about
them. I care only about upstream driver (AMD, Intel, NVidia) and i
will not do an HMM feature if i do not intend to use it in at least
one of those upstream driver. Yes i have work with NVidia on the
design simply because they are the market leader on GPU compute and
have talented engineers that know a little about what would work
well. Not working with them to get their input on design just because
their driver is closed source seems radical to me. I believe i
benefited from their valuable input. But in the end my aim is, and
have always been, to make the upstream kernel driver the best as
possible. I will talk with anyone that can help in achieving that
objective.

So do not worry about non upstream driver.


> [..]
> > Again HMM API can evolve, i am happy to help with any such change, given
> > it provides benefit to either mm or device driver (ie changing the HMM
> > just for the sake of changing the HMM API would not make much sense to
> > me).
> >
> > So if after converting driver A, B and C we see that it would be nicer
> > to change HMM in someway then i will definitly do that and this patchset
> > is a testimony of that. Converting ODP to use HMM is easier after this
> > patchset and this patchset changes the HMM API. I will be updating the
> > nouveau driver to the new API and use the new API for the other driver
> > patchset i am working on.
> >
> > If i bump again into something that would be better done any differently
> > i will definitly change the HMM API and update all upstream driver
> > accordingly.
> >
> > I am a strong believer in full freedom for internal kernel API changes
> > and my intention have always been to help and facilitate such process.
> > I am sorry this was unclear to any body :( and i am hopping that this
> > email make my intention clear.''
> 
> A simple way to ensure that out-of-tree consumers don't come beat us
> up over a backwards incompatible HMM change is to mark all the exports
> with _GPL. I'm not requiring that, the devm_memremap_pages() fight was
> hard enough, but the pace of new exports vs arrival of consumers for
> those exports has me worried that this arrangement will fall over at
> some point.

I was reluctant with the devm_memremap_pages() GPL changes because i
think we should not change symbol export after an initial choice have
been made on those.

I don't think GPL or non GPL export change one bit in respect to out
of tree user. They know they can not make any legitimate regression
claim, nor should we care. So i fail to see how GPL export would make
it any different.

> Another way to help allay these worries is commit to no new exports
> without in-tree users. In general, that should go without saying for
> any core changes for new or future hardware.

I always intend to have an upstream user the issue is that the device
driver tree and the mm tree move a different pace and there is always
a chicken and egg problem. I do not think Andrew wants to have to
merge driver patches through its tree, nor Linus want to have to merge
drivers and mm trees in specific order. So it is easier to introduce
mm change in one release and driver change in the next. This is what
i am doing with ODP. Adding things necessary in 5.1 and working with
Mellanox to have the ODP HMM patch fully tested and ready to go in
5.2 (the patch is available today and Mellanox have begin testing it
AFAIK). So this is the guideline i will be following. Post mm bits
with driver patches, push to merge mm bits one release and have the
driver bits in the next. I do hope this sound fine to everyone.

It is also easier for the driver folks as then they do not need to
have a special tree just to test my changes. They can integrate it
in their regular workflow ie merge the new kernel release in their
tree and then start pilling up changes to their driver for the next
kernel release.

Cheers,
Jérôme