Re: [RFC PATCH 0/5] Add CONFIG symbol as module attribute

[Luis Chamberlain]

On Wed, Jul 03, 2019 at 08:57:58PM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jul 03, 2019 at 04:50:20PM +0000, Luis Chamberlain wrote:
> > On Wed, Jul 03, 2019 at 09:40:48AM +0200, Greg Kroah-Hartman wrote:
> > > On Tue, Jul 02, 2019 at 08:51:06PM +0000, Luis Chamberlain wrote:
> > > > On Sat, Jun 29, 2019 at 10:42:57AM +0200, Greg Kroah-Hartman wrote:
> > > > > On Fri, Jun 28, 2019 at 11:40:22AM -0700, Luis Chamberlain wrote:
> > > > > > On Wed, Jun 26, 2019 at 9:51 PM Christoph Hellwig <hch@xxxxxx> wrote:
> > > > > > >
> > > > > > > On Wed, Jun 26, 2019 at 03:21:08PM -0700, Luis Chamberlain wrote:
> > > > > > > > On Tue, Feb 5, 2019 at 2:07 PM Luis Chamberlain <mcgrof@xxxxxxxxxx> wrote:
> > > > > > > > > In lieu of no Luke Skywalker, if you will, for a large kconfig revamp
> > > > > > > > > on this, I'm inclined to believe *at least* having some kconfig_symb
> > > > > > > > > exposed for some modules is better than nothing. Christoph are you
> > > > > > > > > totally opposed to this effort until we get a non-reverse engineered
> > > > > > > > > effort in place? It just seems like an extraordinary amount of work
> > > > > > > > > and I'm not quite sure who's volunteering to do it.
> > > > > > > > >
> > > > > > > > > Other stakeholders may benefit from at least having some config -->
> > > > > > > > > module mapping for now. Not just backports or building slimmer
> > > > > > > > > kernels.
> > > > > > > >
> > > > > > > > Christoph, *poke*
> > > > > > >
> > > > > > > Yes, I'm still totally opposed to a half-backed hack like this.
> > > > > > 
> > > > > > The solution puts forward a mechanism to add a kconfig_symb where we
> > > > > > are 100% certain we have a direct module --> config mapping.
> > > > > > 
> > > > > > This is *currently* determined when the streamline_config.pl finds
> > > > > > that an object has only *one* associated config symbol associated. As
> > > > > > Cristina noted, of 62 modules on a running system 58 of them ended up
> > > > > > getting the kconfig_symb assigned, that is 93.5% of all modules on the
> > > > > > system being tested. For the other modules, if they did want this
> > > > > > association, we could allow a way for modules to define their own
> > > > > > KBUILD_KCONF variable so that this could be considered as well, or
> > > > > > they can look at their own kconfig stuff to try to fit the model that
> > > > > > does work. To be clear, the heuristics *can* be updated if there is
> > > > > > confidence in alternative methods for resolution. But since it is
> > > > > > reflective of our current situation, I cannot consider it a hack.
> > > > > > 
> > > > > > This implementation is a reflection of our reality in the kernel, and
> > > > > > as has been discussed in this thread, if we want to correct the gaps
> > > > > > we need to do a lot of work. And *no one* is working towards these
> > > > > > goals.
> > > > > > 
> > > > > > That said, even if you go forward with an intrusive solution like the
> > > > > > one you proposed we could still use the same kconfig_symb...
> > > > > > 
> > > > > > So no, I don't see this as a hack. It's a reflection as to our current
> > > > > > reality. And I cannot see how the kconfig_symb can lie or be
> > > > > > incorrect. So in fact I think that pushing this forward also makes the
> > > > > > problem statement clearer for the future of what semantics needs to be
> > > > > > addressed, and helps us even annotate the problematic areas of the
> > > > > > kernel.
> > > > > > 
> > > > > > What negative aspects do you see with this being merged in practice?
> > > > > 
> > > > > I'm trying to see what the actual problem that you are wanting to solve
> > > > > here with this.  What exactly is it?
> > > > 
> > > > The problem is that there is no current maping of a module to respective
> > > > kconfig symbol.
> > > 
> > > That's because it is not just "one" symbol per module.
> > 
> > This is true. But it is not the case for all modules.  In fact it seems
> > its true that most modules do have *one* main symbol.
> 
> You mean "one unique symbol from all other modules", right?

I mean that in the typical case you have *one* main *parent* symbol defining
one device driver for one piece of hardware.

> That is much different than just "one" symbol, given that almost
> every driver depends on something else being enabled as well (bus type,
> platform type, arch, etc.)

Yes but that is different than what is trying to be addressed.

> And I would argue, that finding that "one" symbol is easy, just parse
> the Makefiles.  But I would also state that this "one" symbol doesn't
> really help you much as those are the "simple" things.  It's how to
> turn on all of the required symbols to get to that "one" symbol that is
> the hard part.

Absolutely, but *that* is a different problem! But I am glad you brought
that up and acknowledge it. Addressing that problem will take time and
folks are working towards it. Addressing that problem will still however
not address the problem being addressed here.

> And conversely, if you disable that "one" symbol, does that also mean
> you can disable the symbols it depended on?  If so, how far back?

Right.. 

> And what about functionality?  If my usb-storage device is "enabled" in
> the build, yet all filesystems are not, or the needed dm module is not,
> it is useless.

Yes. The localmodconfig approach is to keep enabled as modules *all* currently
enabled modules, that covers that.

But again, this is a separate problem. The one I am addressing, on
behalf of Cristina, is a subspace, dedicated towards *hardware*
functionality.

> Hardware requires usually more than one real "symbol" in
> order to work properly, as you know.

Right. This does not mean that this information of parent main symbol is
not useful. And having it available on the modules can help with
multiple goals, without requiring kernel sources.

> And of course, what does this really matter to anyone?  If you build
> "all modules" and you only load the modules you actually use for your
> hardware (based on auto-loading), then your system uses the same amount
> of memory as if you disabled all of the modules you did not need.

For backports it means you can enable only what you need, or at least
show users what modules in a backports release could be upgraded and let
you enable / disable them.  For other users, people can care about build
time. And not everyone has fancy systems to build all modules; and
sometimes you may just want to enable a small qemu system and build
locally on it, enabling all modules on such systems would just be
extremely silly.

> Yes, it's faster to build, but is that what you are trying to optimize
> for here?

Particularly for me, yes. Others have other needs and I have already
stated clearly what the gains are.

> Anyway, if this is just an acidemic thing, have fun, but I would not be
> adding anything else to the module image that is not really going to be
> useful to anyone.

The academic consensus is kconfig semantics are poo and we need to do
slowly start addressing this. I believe that striving towards having
one kconfig symbol per hw component can help long term, and in the
meantime, this also will help with the 'make localmodconfig' and
backport users.

> good luck!

This is not about luck. If you don't want to address these problems,
that's fine, but please provide objective considerations rather than
right out rejection without a reasonable basis. Or even better, provide
alternative recommendations.

The alternative Christoph recommended is hugely instrusive, no one is
working on it, and the simple solution proposed in this thread at least
gets us a small step forward in helping to enable a few more users,
while also postulating if we should strive towards having one main kconfig
for each hw component. Since it seems this is already the case, and
there are only a few outliers, this effort should help spot outliers
and address them to help with our semantics.

  Luis