Re: [PATCH v2 0/4] kbuild: resurrect generic header check facility

[Jason Gunthorpe]

On Tue, Apr 08, 2025 at 09:42:36PM +0300, Jani Nikula wrote:
> On Tue, 08 Apr 2025, Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
> > On Tue, Apr 08, 2025 at 11:27:58AM +0300, Jani Nikula wrote:
> >> On Mon, 07 Apr 2025, Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
> >> > On Mon, Apr 07, 2025 at 10:17:40AM +0300, Jani Nikula wrote:
> >> >
> >> >> Even with Jason's idea [1], you *still* have to start small and opt-in
> >> >> (i.e. the patch series at hand). You can't just start off by testing
> >> >> every header in one go, because it's a flag day switch. 
> >> >
> >> > You'd add something like 'make header_check' that does not run
> >> > automatically. Making it run automatically after everything is fixed
> >> > to keep it fixed would be the flag day change. It is how we have
> >> > managed to introduce other warning levels in the past.
> >> 
> >> That approach does not help *me* or drm, i915 and xe in the least. They
> >> are already fixed, and we want a way to keep them fixed. This is how all
> >> of this got started.
> >
> > I imagine you'd include a way to have the 'make header_check' run on
> > some subset of files only, then use that in your CI for the interm.
> >
> >> Your goal may be to make everything self-contained, but AFAICS there is
> >> no agreement on that goal. As long as there's no buy-in to this, it's
> >> not possible fix everything, it's an unreachable goal.
> >
> > I didn't see that. I saw technical problems with the implementation
> > that was presented. I'd be shocked if there was broad opposition to
> > adding missing includes and forward declaration to most headers. It is
> > a pretty basic C thing. :\
> 
> Unless I'm mistaken, both Linus and Masahiro have said they disagree
> with headers having to be self-contained as a general rule, regardless
> of the issues with kconfig and the build artifacts.

Right, no general rule.

But the data I just ran shows the vast majority are already self
contained (~15% are not) and many are trivially fixable to be self
contained. There is a fairly small minority that will not and should
not be self contained.

So I expect there is alot of headers where people would agree to add
the missing #include <linux/types.h> for example, which I found
about 20 of in about 10 mins.

And a smallish exclusion list to ignore the special cases. Ie I
started by just regex ignoring all of asm because there was lots of
interesting stuff in there.

The point is we can probably get to a full kernel check, with a
minority of special headers excluded, that does not have any errors.

As I said in my first email I think this brings real actual value to
people using clangd. AFAICT there is no good reason that every day
normal headers should be missing their #include <linux/types.h> (which
seems to be the most common error)

This is where I think it is constructive to present what the actual
proposed header files changes would be.

> > You can run W=1 using a subdirectory build just for your drivers.
> 
> I don't think there's a way to build the entire kernel while limiting
> W=1 warnings to a subdirectory, is there? Mixing W=1 and regular builds
> causes everything to be rebuilt due to dependencies. It's not only for
> CI, it's also for developers.

You'd have to do the W=0 build then a subdirectory W=1 build.

I agree this is annoying and I do wish kbuild had a better solution
here.

> Thanks for the proof-of-concept. It's just that I don't see how that
> could be bolted to kbuild, with dependency tracking. I don't want to
> have to rebuild the world every time something changes.

I used ninja to run this because it is very easy to get setup and
going and doesn't leave behind the 'turds'. The main point was to show
that the .cmd processing and so on works sensibly and does avoid the
kconfig issues.

If people agree to stick with ninja for this then you'd use the -MD
option to gcc and the depfile=foo.d instruction then you get full
dependency tracking and incremental compilation. Along with a rule to
rebuild the rule file if any .cmd file changes. I did not show this,
but it is very easy.

Jason