Re: [RFC PATCH 0/5] Add CONFIG symbol as module attribute

[Brendan Higgins]

On Wed, Jul 3, 2019 at 3:25 PM Luis Chamberlain <mcgrof@xxxxxxxxxx> wrote:
>
> On Wed, Jul 03, 2019 at 08:57:58PM +0200, Greg Kroah-Hartman wrote:
> > On Wed, Jul 03, 2019 at 04:50:20PM +0000, Luis Chamberlain wrote:
> > > On Wed, Jul 03, 2019 at 09:40:48AM +0200, Greg Kroah-Hartman wrote:
> > > > On Tue, Jul 02, 2019 at 08:51:06PM +0000, Luis Chamberlain wrote:
> > > > > On Sat, Jun 29, 2019 at 10:42:57AM +0200, Greg Kroah-Hartman wrote:
> > > > > > On Fri, Jun 28, 2019 at 11:40:22AM -0700, Luis Chamberlain wrote:
> > > > > > > On Wed, Jun 26, 2019 at 9:51 PM Christoph Hellwig <hch@xxxxxx> wrote:
> > > > > > > >
> > > > > > > > On Wed, Jun 26, 2019 at 03:21:08PM -0700, Luis Chamberlain wrote:
> > > > > > > > > On Tue, Feb 5, 2019 at 2:07 PM Luis Chamberlain <mcgrof@xxxxxxxxxx> wrote:
> > > > > > > > > > In lieu of no Luke Skywalker, if you will, for a large kconfig revamp
> > > > > > > > > > on this, I'm inclined to believe *at least* having some kconfig_symb
> > > > > > > > > > exposed for some modules is better than nothing. Christoph are you
> > > > > > > > > > totally opposed to this effort until we get a non-reverse engineered
> > > > > > > > > > effort in place? It just seems like an extraordinary amount of work
> > > > > > > > > > and I'm not quite sure who's volunteering to do it.
> > > > > > > > > >
> > > > > > > > > > Other stakeholders may benefit from at least having some config -->
> > > > > > > > > > module mapping for now. Not just backports or building slimmer
> > > > > > > > > > kernels.
> > > > > > > > >
> > > > > > > > > Christoph, *poke*
> > > > > > > >
> > > > > > > > Yes, I'm still totally opposed to a half-backed hack like this.
> > > > > > >
> > > > > > > The solution puts forward a mechanism to add a kconfig_symb where we
> > > > > > > are 100% certain we have a direct module --> config mapping.
> > > > > > >
> > > > > > > This is *currently* determined when the streamline_config.pl finds
> > > > > > > that an object has only *one* associated config symbol associated. As
> > > > > > > Cristina noted, of 62 modules on a running system 58 of them ended up
> > > > > > > getting the kconfig_symb assigned, that is 93.5% of all modules on the
> > > > > > > system being tested. For the other modules, if they did want this
> > > > > > > association, we could allow a way for modules to define their own
> > > > > > > KBUILD_KCONF variable so that this could be considered as well, or
> > > > > > > they can look at their own kconfig stuff to try to fit the model that
> > > > > > > does work. To be clear, the heuristics *can* be updated if there is
> > > > > > > confidence in alternative methods for resolution. But since it is
> > > > > > > reflective of our current situation, I cannot consider it a hack.
> > > > > > >
> > > > > > > This implementation is a reflection of our reality in the kernel, and
> > > > > > > as has been discussed in this thread, if we want to correct the gaps
> > > > > > > we need to do a lot of work. And *no one* is working towards these
> > > > > > > goals.
> > > > > > >
> > > > > > > That said, even if you go forward with an intrusive solution like the
> > > > > > > one you proposed we could still use the same kconfig_symb...
> > > > > > >
> > > > > > > So no, I don't see this as a hack. It's a reflection as to our current
> > > > > > > reality. And I cannot see how the kconfig_symb can lie or be
> > > > > > > incorrect. So in fact I think that pushing this forward also makes the
> > > > > > > problem statement clearer for the future of what semantics needs to be
> > > > > > > addressed, and helps us even annotate the problematic areas of the
> > > > > > > kernel.
> > > > > > >
> > > > > > > What negative aspects do you see with this being merged in practice?
> > > > > >
> > > > > > I'm trying to see what the actual problem that you are wanting to solve
> > > > > > here with this.  What exactly is it?
> > > > >
> > > > > The problem is that there is no current maping of a module to respective
> > > > > kconfig symbol.
> > > >
> > > > That's because it is not just "one" symbol per module.
> > >
> > > This is true. But it is not the case for all modules.  In fact it seems
> > > its true that most modules do have *one* main symbol.
> >
> > You mean "one unique symbol from all other modules", right?
>
> I mean that in the typical case you have *one* main *parent* symbol defining
> one device driver for one piece of hardware.
>
> > That is much different than just "one" symbol, given that almost
> > every driver depends on something else being enabled as well (bus type,
> > platform type, arch, etc.)
>
> Yes but that is different than what is trying to be addressed.
>
> > And I would argue, that finding that "one" symbol is easy, just parse
> > the Makefiles.  But I would also state that this "one" symbol doesn't
> > really help you much as those are the "simple" things.  It's how to
> > turn on all of the required symbols to get to that "one" symbol that is
> > the hard part.
>
> Absolutely, but *that* is a different problem! But I am glad you brought
> that up and acknowledge it. Addressing that problem will take time and
> folks are working towards it. Addressing that problem will still however
> not address the problem being addressed here.
>
> > And conversely, if you disable that "one" symbol, does that also mean
> > you can disable the symbols it depended on?  If so, how far back?
>
> Right..
>
> > And what about functionality?  If my usb-storage device is "enabled" in
> > the build, yet all filesystems are not, or the needed dm module is not,
> > it is useless.
>
> Yes. The localmodconfig approach is to keep enabled as modules *all* currently
> enabled modules, that covers that.
>
> But again, this is a separate problem. The one I am addressing, on
> behalf of Cristina, is a subspace, dedicated towards *hardware*
> functionality.

Hmm...maybe this isn't what I am looking for then. I am interested in
the problem of figuring out what dependencies I need to select to turn
on a desired config symbol (which is obviously a separate issue), and
I am interested in associating symbols with a config symbol and then
ensuring that symbol is exercised. Basically, I want a way to make
sure my tests actually get run without a human looking at them; I feel
like what you are working on might help with this latter issue, but I
am not 100% sure. It sounds like it is not your primary goal in any
case.

> > Hardware requires usually more than one real "symbol" in
> > order to work properly, as you know.
>
> Right. This does not mean that this information of parent main symbol is
> not useful. And having it available on the modules can help with
> multiple goals, without requiring kernel sources.
>
> > And of course, what does this really matter to anyone?  If you build
> > "all modules" and you only load the modules you actually use for your
> > hardware (based on auto-loading), then your system uses the same amount
> > of memory as if you disabled all of the modules you did not need.
>
> For backports it means you can enable only what you need, or at least
> show users what modules in a backports release could be upgraded and let
> you enable / disable them.  For other users, people can care about build
> time. And not everyone has fancy systems to build all modules; and
> sometimes you may just want to enable a small qemu system and build
> locally on it, enabling all modules on such systems would just be
> extremely silly.
>
> > Yes, it's faster to build, but is that what you are trying to optimize
> > for here?
>
> Particularly for me, yes. Others have other needs and I have already
> stated clearly what the gains are.
>
> > Anyway, if this is just an acidemic thing, have fun, but I would not be
> > adding anything else to the module image that is not really going to be
> > useful to anyone.
>
> The academic consensus is kconfig semantics are poo and we need to do
> slowly start addressing this. I believe that striving towards having
> one kconfig symbol per hw component can help long term, and in the
> meantime, this also will help with the 'make localmodconfig' and
> backport users.
>
> > good luck!
>
> This is not about luck. If you don't want to address these problems,
> that's fine, but please provide objective considerations rather than
> right out rejection without a reasonable basis. Or even better, provide
> alternative recommendations.
>
> The alternative Christoph recommended is hugely instrusive, no one is
> working on it, and the simple solution proposed in this thread at least
> gets us a small step forward in helping to enable a few more users,
> while also postulating if we should strive towards having one main kconfig
> for each hw component. Since it seems this is already the case, and
> there are only a few outliers, this effort should help spot outliers
> and address them to help with our semantics.

Ooo, looks like I should take a look at that.

Cheers!