Re: [PROBLEM] Very long .deb package build times for bindeb-pkg build target

[Mirsad Todorovac]

On 11. 01. 2024. 16:37, Nicolas Schier wrote:
> Hi Mirsad,
> 
> On Thu 11 Jan 2024 13:22:39 GMT, Mirsad Todorovac wrote:
>> Hi,
>>
>> With this new release, it seems that Debian kernel build uses "xz" in single-
>> threaded mode:
>>
>> Tasks: 484 total,   2 running, 481 sleeping,   0 stopped,   1 zombie
>> %Cpu(s):  2.5 us,  2.2 sy,  6.3 ni, 85.1 id,  2.3 wa,  0.0 hi,  1.7 si,  0.0 st
>> MiB Mem :  64128.3 total,    524.3 free,   5832.0 used,  58540.9 buff/cache
>> MiB Swap:  32760.0 total,  32758.7 free,      1.2 used.  58296.3 avail Mem
>>
>>     PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+
>> COMMAND
>>
>>  978084 marvin    30  10  112440  97792   2432 R 100.0   0.1  29:30.23 xz
>>
>>
>> Before dpkg-deb was using up to 3200% of CPU time on a 16 core SMT CPU.
>>
>> Can it be something with dpkg-deb --thread-max=%n option?
> 
> I cannot find any --thread-max option in Linux tree.  Do you call 
> dpkg-deb manually or somehow induce a thread maximum?
>  
>> Waiting for half an hour just for the build of linux-image-...-dbg package
>> seems like an overkill ...
> 
> With current v6.7 release tree I do not see the reported slow-downs 
> when building bindeb-pkg; I tested by cross-compiling for arm64 on 
> amd64 with CONFIG_MODULE_COMPRESS_XZ=y and =n).
> 
> Both take roughly 5mins on my 24-core i9 system.
> 
> Kind regards,
> Nicolas

I am perplexed too, but you can see from the top output the
single-threaded xz with 29:30m processor time.

On my laptop with the sam Ubuntu 23.10 mantic minotaur, I have
dpkg-deb version 1.20.12 and it shows things like 400% and 3200%
CPU time, so it is working multithreaded.

On desktop machine with the same Ubuntu 23.10 and the same git
torvalds tree, it starts single-threaded xz from dpkg-deb instead.

I tried things like this:

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index d7dd0d04c70c..b2319c23db34 100755
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -38,7 +38,7 @@ create_package() {
 
        # Fix ownership and permissions
        if [ "$DEB_RULES_REQUIRES_ROOT" = "no" ]; then
-               dpkg_deb_opts="--root-owner-group"
+               dpkg_deb_opts="--threads-max=0 --root-owner-group"
        else
                chown -R root:root "$pdir"
        fi

and it didn't work either - dpkg-deb --threads-max=0 still spawned a
single-threaded xz that ran 30 minutes.

Then the workaround was a very simple xz shell script that adds option --threads=0 
and calls system xz:

~/bin/xz:
----------------------------------------------------------------
#!/bin/bash -f

/usr/bin/xz --threads=0 "$@"
----------------------------------------------------------------

This finally worked, but sometimes I get:

marvin@defiant:~/linux/kernel$ xz -9 --memlimit-compress=8000MiB linux-image-6.7.0-rc8-dbg_6.7.0-rc8-6_amd64.deb
/usr/bin/xz: Reduced the number of threads from 8 to 6 to not exceed the memory usage limit of 8000 MiB

(This is of course just an example of compressing a large file, as .deb is already compressed.)

I used the default Ubuntu 23.10 config, with .pems excluded, and I think module compression
did not work either. I had to turn it off ...

Hope this helps.

Regards,
Mirsad

-- 
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
 
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union

"I see something approaching fast ... Will it be friends with me?"