Re: Parallel compilation performance regression

[Sedat Dilek]

On Sat, Jun 20, 2020 at 5:53 AM Derrick, Jonathan
<jonathan.derrick@xxxxxxxxx> wrote:
>
> On Thu, 2020-06-18 at 18:05 -0700, Matthew Wilcox wrote:
> > On Thu, Jun 18, 2020 at 11:52:55PM +0000, Derrick, Jonathan wrote:
> > > Hi David,
> > >
> > > I've been experiencing a performance regression when running a parallel
> > > compilation (eg, make -j72) on recent kernels.
> >
> > I bet you're using a version of make which predates 4.3:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0ddad21d3e99c743a3aa473121dc5561679e26bb
> >
>
> I am!
>
> # make --version
> GNU Make 4.2.1
>
>
> Thank you Matthew!

Check your distribution and included patches on top of a vanilla make v4.2.
Debian had some important ones I described in [1].

When proposing "make version 4.3" for tc-build - an opportunity to
build a llvm-toolchain the easy way - we saw different numbers.
I pointed in [1] to the Linus patch Matthew did here.

Personally, with switching to Debian's make version 4.3-3 I have seen
no big differences when using "make -j3" to build Linux v5.7+.
That might be different with "make -j72"...
...can I have SSH access to this machine, please :-),

You forgot to tell which Linux version you use.

If you are interested please look at closed tc-build issue #72 for our analysis.
For tc-build it did not matter - (Debian's) make v4.2 had some
slightly better performance.

BTW, with that pipe improvements in Linux v5.7 I see some better
numbers when using pipebench to benchmark my devices:

Example: SanDisk iSSD 16GB

root# cat /dev/sdb | pipebench > /dev/null
Summary:
Piped   14.91 GB in 00h01m25.20s:  179.23 MB/second

Before: approx. 100MB/s

BTW, I heard of hyperfine benchmark tool the first time when dealing
with make performance (see [2]) in ClangBuiltLinux.
I would like to see some benchmark numbers with hyperfine from you if
you do not mind.

Stay OPEN minded and curious.

Thanks.

- Sedat -

[1] https://github.com/ClangBuiltLinux/tc-build/issues/72
[2] https://github.com/sharkdp/hyperfine
[3] https://github.com/sharkdp/hyperfine/releases