Re: [PATCH v2 14/21] kbuild: batch module finalisation

[Kees Cook]

On Tue, Sep 15, 2026 at 11:44:01AM +0100, Lorenzo Stoakes (ARM) wrote:
> On Mon, Sep 14, 2026 at 11:00:27AM -0700, Kees Cook wrote:
> > On Mon, Sep 14, 2026 at 10:22:13AM +0100, Lorenzo Stoakes (ARM) wrote:
> > > Module finalisation on allmodconfig builds consists of a large number of
> > > very short-lived jobs, and the make job dispatcher cannot possibly dispatch
> > > jobs fast enough.
> >
> > This "cannot possibly" sounds like a weird LLM language intensifier. I'd
> 
> Again that's my writing actually, probably best to stop assuming LLM now ;)

Apologies; whatever the opposite of the Turing Test is, I have failed.
:P The AIs are winning: humans are suspicious of each other now. :)

> > rather a concrete description of the problem, not this kind of
> > (redundant?) vagueness.
> 
> In the very next paragraph I say:
> 
> 	For allmodconfig x86-64 this can be on the order of ~22,000 jobs of
> 	a few milliseconds in duration each.
> 
> You did also say that the commit messages were over-long, so there's a
> trade-off here :)

Right, sorry, I meant I understand the need to collapse the jobs
together, I was just trying to understand where the "128" came from.

> But in more detail, allmodconfig tree (~11k modules on x86-64), best of 2,
> make modules with *.ko *.mod.o deleted:
> 
>     modules per chunk    instances    wall
>     -----------------    ---------    ------
>                     1        11171    10.22s
>                     2         5586     6.72s
>                     4         2793     4.97s
>                     8         1397     4.26s
>                    16          699     4.05s
>                    32          350     4.05s
>                    64          175     4.07s
>                   128           88     4.05s
>                   256           44     4.03s
>                   512           22     4.18s
> 
> which I think makes things clearer.
> 
> So realistically 32 - 256 is the right sort of range. You also have to take
> into account the fact that you might be building fewer modules.

This answers it, thank you!

> The argument for 128 is that this is the mid-point of where the graph
> flattens off for a larger number of modules.
> 
> For a smaller number, you're going to have a single dispatch or less and
> the delta won't be that much anyway.

Understood. I guess I'm trying to better understand the chosen balance
point. Does machine CPU count matter at all for this? i.e. "16" is the
first one to hit the 4 second line, can that be "better" than 128 for
smaller system (or fewer modules)? I'm trying to consider CIs and such
that are bounded to 8 or fewer threads, etc.

> So it's very much a sensibly derived empirical value.
> 
> I'll update the commit message to give this rationale there.

Thanks!

-Kees

-- 
Kees Cook