Re: [PATCH 17/23] modpost: perform srcversion hashing in parallel

[Lorenzo Stoakes (ARM)]

On Sat, Sep 12, 2026 at 01:50:00PM +0200, Yann Droneaud wrote:
> Hi,
>
> Le 08/09/2026 à 22:55, Lorenzo Stoakes (ARM) a écrit :
> > modpost does a lot of single-threaded work hashing files from each object's
> > .cmd file.
> >
> > This makes the build slower than it needs to be, so do this work in
> > parallel.
>
>
> If you're going parallel, it would be nicer to do it under make's jobserver
> supervision.

Ack, though this patch is going to be dropped due to mod versions being removed
upstream.

>
>
> > This is egregious for allmodconfig builds - for instance x86-64 can end up
> > opening 200,000 files individually and hashing them all serially.
> >
> > Parallelise this operation by maintaining a thread pool for the hashing
> > work.
> >
> > Combined with the per-file hashing commit this cuts modpost's run time
> > nearly in half for an allmodconfig build.
> >
> > Module.symvers and every *.mod.S are byte for byte the same.
> >
> > modpost is on the serial tail of every allmodconfig build, however
> > defconfig does not set CONFIG_MODULE_SRCVERSION_ALL and is unchanged.
> >
> > Whole build, 128-thread Threadripper 9980X, best of N runs:
> >
> >                                           before   after     delta
> >                                           -------------------------------
> >    x86 allmodconfig, touch mm/vma.c, gcc    33.4s    30.2s     -3.2s (-10%)
> >    x86 allmodconfig, touch mm/vma.c, clang  31.1s    28.1s     -3.0s (-10%)
> >
> > Assisted-by: LLM
> > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>
> > ---
> >   scripts/mod/Makefile     |  1 +
> >   scripts/mod/modpost.c    | 72 ++++++++++++++++++++++++++++++++++++++++++++++--
> >   scripts/mod/modpost.h    |  2 ++
> >   scripts/mod/sumversion.c |  3 +-
> >   4 files changed, 74 insertions(+), 4 deletions(-)
> >
>
> Regards.
>
>

--
Cheers, Lorenzo