Re: [PATCH v2 3/2] sched/deadline: Check bandwidth overflow earlier for hotplug
From: Jon Hunter
Date: Wed Feb 05 2025 - 11:56:55 EST
Hi Juri,
On 05/02/2025 10:12, Juri Lelli wrote:
On 05/02/25 07:53, Juri Lelli wrote:
On 03/02/25 11:01, Jon Hunter wrote:
Hi Juri,
On 16/01/2025 15:55, Juri Lelli wrote:
On 16/01/25 13:14, Jon Hunter wrote:
...
[ 210.595431] dl_bw_manage: cpu=5 cap=3072 fair_server_bw=52428 total_bw=209712 dl_bw_cpus=4
[ 210.606269] dl_bw_manage: cpu=4 cap=2048 fair_server_bw=52428 total_bw=157284 dl_bw_cpus=3
[ 210.617281] dl_bw_manage: cpu=3 cap=1024 fair_server_bw=52428 total_bw=104856 dl_bw_cpus=2
[ 210.627205] dl_bw_manage: cpu=2 cap=1024 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=2
[ 210.637752] dl_bw_manage: cpu=1 cap=0 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=1
^
Different than before but still not what I expected. Looks like there
are conditions/path I currently cannot replicate on my setup, so more
thinking. Unfortunately I will be out traveling next week, so this
might required a bit of time.
I see that this is now in the mainline and our board is still failing to
suspend. Let me know if there is anything else you need me to test.
Ah, can you actually add 'sched_verbose' and to your kernel cmdline? It
should print our additional debug info on the console when domains get
reconfigured by hotplug/suspends, e.g.
dl_bw_manage: cpu=3 cap=3072 fair_server_bw=52428 total_bw=209712 dl_bw_cpus=4
CPU0 attaching NULL sched-domain.
CPU3 attaching NULL sched-domain.
CPU4 attaching NULL sched-domain.
CPU5 attaching NULL sched-domain.
CPU0 attaching sched-domain(s):
domain-0: span=0,4-5 level=MC
groups: 0:{ span=0 cap=766 }, 4:{ span=4 cap=908 }, 5:{ span=5 cap=989 }
CPU4 attaching sched-domain(s):
domain-0: span=0,4-5 level=MC
groups: 4:{ span=4 cap=908 }, 5:{ span=5 cap=989 }, 0:{ span=0 cap=766 }
CPU5 attaching sched-domain(s):
domain-0: span=0,4-5 level=MC
groups: 5:{ span=5 cap=989 }, 0:{ span=0 cap=766 }, 4:{ span=4 cap=908 }
root domain span: 0,4-5
rd 0,4-5: Checking EAS, CPUs do not have asymmetric capacities
psci: CPU3 killed (polled 0 ms)
Can you please share this information as well if you are able to collect
it (while still running with my last proposed fix)?
Also, if you don't mind, add the following on top of the existing
changes.
Just to be sure we don't get out of sync, I pushed current set to
https://github.com/jlelli/linux.git experimental/dl-debug
Thanks! That did make it easier :-)
Here is what I see ...
[ 53.823979] PM: suspend entry (deep)
[ 53.827715] Filesystems sync: 0.000 seconds
[ 53.832859] Freezing user space processes
[ 53.838132] Freezing user space processes completed (elapsed 0.001 seconds)
[ 53.845118] OOM killer disabled.
[ 53.848348] Freezing remaining freezable tasks
[ 53.853884] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
[ 53.900686] tegra-xusb 3530000.usb: Firmware timestamp: 2020-07-06 13:39:28 UTC
[ 53.918492] dwc-eth-dwmac 2490000.ethernet eth0: Link is Down
[ 53.962316] Disabling non-boot CPUs ...
[ 53.966192] dl_bw_manage: cpu=5 cap=3072 fair_server_bw=52428 total_bw=209712 dl_bw_cpus=4 type=DYN span=0,3-5
[ 53.966231] CPU0 attaching NULL sched-domain.
[ 53.980574] span=1-2
[ 53.982767] CPU3 attaching NULL sched-domain.
[ 53.987119] span=0-2
[ 53.989309] CPU4 attaching NULL sched-domain.
[ 53.993662] span=0-3
[ 53.995853] CPU5 attaching NULL sched-domain.
[ 54.000206] span=0-4
[ 54.002433] CPU0 attaching sched-domain(s):
[ 54.006614] domain-0: span=0,3-4 level=MC
[ 54.010711] groups: 0:{ span=0 cap=1022 }, 3:{ span=3 cap=1022 }, 4:{ span=4 }
[ 54.018126] CPU3 attaching sched-domain(s):
[ 54.022307] domain-0: span=0,3-4 level=MC
[ 54.026404] groups: 3:{ span=3 cap=1022 }, 4:{ span=4 }, 0:{ span=0 cap=1023 }
[ 54.033821] CPU4 attaching sched-domain(s):
[ 54.038001] domain-0: span=0,3-4 level=MC
[ 54.042098] groups: 4:{ span=4 }, 0:{ span=0 cap=1023 }, 3:{ span=3 cap=1022 }
[ 54.049508] root domain span: 0,3-4
[ 54.052997] default domain span: 1-2,5
[ 54.056756] rd 0,3-4: Checking EAS, CPUs do not have asymmetric capacities
[ 54.064688] psci: CPU5 killed (polled 0 ms)
[ 54.069495] dl_bw_manage: cpu=4 cap=2048 fair_server_bw=52428 total_bw=157284 dl_bw_cpus=3 type=DYN span=0,3-4
[ 54.069547] CPU0 attaching NULL sched-domain.
[ 54.083910] span=1-2,5
[ 54.086277] CPU3 attaching NULL sched-domain.
[ 54.090633] span=0-2,5
[ 54.092999] CPU4 attaching NULL sched-domain.
[ 54.097351] span=0-3,5
[ 54.099756] CPU0 attaching sched-domain(s):
[ 54.103941] domain-0: span=0,3 level=MC
[ 54.107865] groups: 0:{ span=0 }, 3:{ span=3 cap=1023 }
[ 54.113279] CPU3 attaching sched-domain(s):
[ 54.117459] domain-0: span=0,3 level=MC
[ 54.121382] groups: 3:{ span=3 cap=1023 }, 0:{ span=0 }
[ 54.126793] root domain span: 0,3
[ 54.130109] default domain span: 1-2,4-5
[ 54.134040] rd 0,3: Checking EAS, CPUs do not have asymmetric capacities
[ 54.141597] psci: CPU4 killed (polled 0 ms)
[ 54.146819] dl_bw_manage: cpu=3 cap=1024 fair_server_bw=52428 total_bw=104856 dl_bw_cpus=2 type=DYN span=0,3
[ 54.156727] CPU0 attaching NULL sched-domain.
[ 54.161086] span=1-2,4-5
[ 54.163632] CPU3 attaching NULL sched-domain.
[ 54.167988] span=0-2,4-5
[ 54.170553] CPU0 attaching NULL sched-domain.
[ 54.174909] span=0-5
[ 54.177096] root domain span: 0
[ 54.180239] default domain span: 1-5
[ 54.183821] rd 0: Checking EAS, CPUs do not have asymmetric capacities
[ 54.191728] psci: CPU3 killed (polled 4 ms)
[ 54.196389] dl_bw_manage: cpu=2 cap=1024 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=2 type=DEF span=1-5
[ 54.196518] rd 0: Checking EAS, CPUs do not have asymmetric capacities
[ 54.214816] psci: CPU2 killed (polled 0 ms)
[ 54.219411] dl_bw_manage: cpu=1 cap=0 fair_server_bw=52428 total_bw=262140 dl_bw_cpus=1 type=DEF span=1-5
[ 54.219493] Error taking CPU1 down: -16
[ 54.232948] Non-boot CPUs are not disabled
[ 54.237046] Enabling non-boot CPUs ...
[ 54.241216] Detected PIPT I-cache on CPU2
[ 54.245258] CPU features: SANITY CHECK: Unexpected variation in SYS_CTR_EL0. Boot CPU: 0x0000008444c004, CPU2: 0x0000009444c004
[ 54.256744] CPU features: SANITY CHECK: Unexpected variation in SYS_ID_AA64DFR0_EL1. Boot CPU: 0x00000010305106, CPU2: 0x00000010305116
[ 54.268954] CPU features: SANITY CHECK: Unexpected variation in SYS_ID_DFR0_EL1. Boot CPU: 0x00000003010066, CPU2: 0x00000003001066
[ 54.280865] CPU2: Booted secondary processor 0x0000000001 [0x4e0f0030]
[ 54.288270] rd 0: Checking EAS, CPUs do not have asymmetric capacities
[ 54.295061] CPU2 is up
[ 54.297599] Detected PIPT I-cache on CPU3
[ 54.301642] CPU3: Booted secondary processor 0x0000000101 [0x411fd073]
[ 54.308419] CPU0 attaching NULL sched-domain.
[ 54.312786] span=1-5
[ 54.315031] CPU0 attaching sched-domain(s):
[ 54.319220] domain-0: span=0,3 level=MC
[ 54.323145] groups: 0:{ span=0 }, 3:{ span=3 cap=1016 }
[ 54.328564] CPU3 attaching sched-domain(s):
[ 54.332746] domain-0: span=0,3 level=MC
[ 54.336671] groups: 3:{ span=3 cap=1016 }, 0:{ span=0 }
[ 54.342080] root domain span: 0,3
[ 54.345405] default domain span: 1-2,4-5
[ 54.349338] rd 0,3: Checking EAS, CPUs do not have asymmetric capacities
[ 54.356122] CPU3 is up
[ 54.358649] Detected PIPT I-cache on CPU4
[ 54.362677] CPU4: Booted secondary processor 0x0000000102 [0x411fd073]
[ 54.369399] CPU0 attaching NULL sched-domain.
[ 54.373767] span=1-2,4-5
[ 54.376310] CPU3 attaching NULL sched-domain.
[ 54.380667] span=0-2,4-5
[ 54.383251] CPU0 attaching sched-domain(s):
[ 54.387439] domain-0: span=0,3-4 level=MC
[ 54.391538] groups: 0:{ span=0 }, 3:{ span=3 cap=1021 }, 4:{ span=4 }
[ 54.398173] CPU3 attaching sched-domain(s):
[ 54.402356] domain-0: span=0,3-4 level=MC
[ 54.406456] groups: 3:{ span=3 cap=1021 }, 4:{ span=4 }, 0:{ span=0 }
[ 54.413090] CPU4 attaching sched-domain(s):
[ 54.417271] domain-0: span=0,3-4 level=MC
[ 54.421373] groups: 4:{ span=4 }, 0:{ span=0 }, 3:{ span=3 cap=1021 }
[ 54.428005] root domain span: 0,3-4
[ 54.431503] default domain span: 1-2,5
[ 54.435259] rd 0,3-4: Checking EAS, CPUs do not have asymmetric capacities
[ 54.442287] CPU4 is up
[ 54.444821] Detected PIPT I-cache on CPU5
[ 54.448848] CPU5: Booted secondary processor 0x0000000103 [0x411fd073]
[ 54.455574] CPU0 attaching NULL sched-domain.
[ 54.459950] span=1-2,5
[ 54.462315] CPU3 attaching NULL sched-domain.
[ 54.466674] span=0-2,5
[ 54.469042] CPU4 attaching NULL sched-domain.
[ 54.473401] span=0-3,5
[ 54.475812] CPU0 attaching sched-domain(s):
[ 54.480000] domain-0: span=0,3-5 level=MC
[ 54.484099] groups: 0:{ span=0 }, 3:{ span=3 }, 4:{ span=4 }, 5:{ span=5 }
[ 54.491171] CPU3 attaching sched-domain(s):
[ 54.495352] domain-0: span=0,3-5 level=MC
[ 54.499452] groups: 3:{ span=3 }, 4:{ span=4 }, 5:{ span=5 }, 0:{ span=0 }
[ 54.506519] CPU4 attaching sched-domain(s):
[ 54.510703] domain-0: span=0,3-5 level=MC
[ 54.514800] groups: 4:{ span=4 }, 5:{ span=5 }, 0:{ span=0 }, 3:{ span=3 }
[ 54.521869] CPU5 attaching sched-domain(s):
[ 54.526050] domain-0: span=0,3-5 level=MC
[ 54.530150] groups: 5:{ span=5 }, 0:{ span=0 }, 3:{ span=3 }, 4:{ span=4 }
[ 54.537217] root domain span: 0,3-5
[ 54.540716] default domain span: 1-2
[ 54.544303] rd 0,3-5: Checking EAS, CPUs do not have asymmetric capacities
[ 54.551393] rd 0,3-5: Checking EAS, CPUs do not have asymmetric capacities
[ 54.558281] CPU5 is up
[ 54.568000] dwc-eth-dwmac 2490000.ethernet eth0: configuring for phy/rgmii link mode
[ 55.585391] dwc-eth-dwmac 2490000.ethernet: Failed to reset the dma
[ 55.591664] dwc-eth-dwmac 2490000.ethernet eth0: stmmac_hw_setup: DMA engine initialization failed
[ 55.600905] dwc-eth-dwmac 2490000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 55.615015] usb-conn-gpio 3520000.padctl:ports:usb2-0:connector: repeated role: device
[ 55.617967] tegra-xusb 3530000.usb: Firmware timestamp: 2020-07-06 13:39:28 UTC
[ 55.655665] OOM killer enabled.
[ 55.658813] Restarting tasks ... done.
[ 55.664082] random: crng reseeded on system resumption
[ 55.664403] VDDIO_SDMMC3_AP: voltage operation not allowed
[ 55.674862] PM: suspend exit
--
nvpublic