Re: sched/fair: DELAY_DEQUEUE causes ~25% pipe IPC regression on Raspberry Pi 5

[Mike Galbraith]

On Mon, 2026-04-20 at 12:59 +0200, Tom Gebhardt wrote:
> Hi Mike,
> 
> thank you for testing — but I notice that your data only shows 6.12.75
> with DELAY_DEQUEUE on/off. There is no 6.6 baseline in your results.
> That is the comparison that matters: 6.6 vs 6.12 on the same hardware
> with the same tool.
> 
> There is also a version difference worth noting: you are running
> stress-ng 0.21.00, while our measurements used stress-ng 0.15.06 —
> which was the version available on Raspberry Pi OS Bookworm at the
> time of the original report. The pipe stressor calculation changed
> significantly between those versions.
> 
> This creates an uncomfortable coincidence: a measurable scheduler
> regression was introduced in 6.12 (DELAY_DEQUEUE), and around the same
> time the standard measurement tool changed how it calculates the pipe
> benchmark score. The result is that the regression becomes invisible
> when comparing with the newer tool version — not because it was fixed,
> but because the metric changed.

Aha.

> Would you be willing to run the 6.6 comparison with stress-ng 0.15.06
> on your Pi 5?

No need, your reported DELAY_DEQUEUE impact for 6.12.75+rpt-rpi-2712
appeared with stress-ng 0.15.06.

However, my local 6.12.y branch kernel, which has all eevdf fixes as
well as Peter's still pending sched/ttwu series, still does NOT repro.

git@homer:..git/raspberrypi-kernel> quilt applied|grep delay
patches/eevdf/WIP/0043-sched-fair-Removed-unsued-cfs_rq.h_nr_delayed.patch
patches/eevdf/WIP/0047-sched-fair-Do-not-try-to-migrate-delayed-dequeue-task.patch           <== ?
patches/eevdf/WIP/0065-sched-Change-ttwu_runnable-vs-sched_delayed.patch
patches/eevdf/WIP/0066-sched-Add-ttwu_queue-support-for-delayed-tasks.patch
patches/eevdf/WIP/0089-sched-fair-Prevent-negative-lag-increase-during-delayed-dequeue.patch <== ?

	-Mike

aside: benchmark preemption model sensitivity magnitude poked me in the
eye while testing, numbers below in case that's of interest to anyone.

preempt=voluntary
rpi5:..debug/sched # /usr/bin/stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2566] setting to a 20 second run per stressor
stress-ng: info:  [2566] dispatching hogs: 4 pipe
stress-ng: metrc: [2566] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2566]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2566] pipe           45294217     20.00     22.71     57.23   2264640.83      566587.74
stress-ng: metrc: [2566] miscellaneous metrics:
stress-ng: metrc: [2566] pipe                 276.48 MB per sec pipe write rate (geometic mean of 4 instances)
stress-ng: info:  [2566] successful run completed in 20.01s
rpi5:..debug/sched # echo NO_DELAY_DEQUEUE > features
rpi5:..debug/sched # /usr/bin/stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2650] setting to a 20 second run per stressor
stress-ng: info:  [2650] dispatching hogs: 4 pipe
stress-ng: metrc: [2650] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2650]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2650] pipe           45379684     20.00     22.69     57.24   2268917.79      567687.53
stress-ng: metrc: [2650] miscellaneous metrics:
stress-ng: metrc: [2650] pipe                 276.99 MB per sec pipe write rate (geometic mean of 4 instances)
stress-ng: info:  [2650] successful run completed in 20.00s
rpi5:..debug/sched # echo DELAY_DEQUEUE > features
rpi5:..debug/sched # /usr/bin/stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2721] setting to a 20 second run per stressor
stress-ng: info:  [2721] dispatching hogs: 4 pipe
stress-ng: metrc: [2721] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2721]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2721] pipe           45217909     20.00     21.81     58.13   2260821.48      565698.97
stress-ng: metrc: [2721] miscellaneous metrics:
stress-ng: metrc: [2721] pipe                 276.00 MB per sec pipe write rate (geometic mean of 4 instances)
stress-ng: info:  [2721] successful run completed in 20.00s
rpi5:..debug/sched #

preempt=lazy
rpi5:..debug/sched # /usr/bin/stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2070] setting to a 20 second run per stressor
stress-ng: info:  [2070] dispatching hogs: 4 pipe
stress-ng: metrc: [2070] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2070]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2070] pipe           32706095     20.00     17.88     62.02   1635263.85      409343.56
stress-ng: metrc: [2070] miscellaneous metrics:
stress-ng: metrc: [2070] pipe                 199.63 MB per sec pipe write rate (geometic mean of 4 instances)
stress-ng: info:  [2070] successful run completed in 20.00s
rpi5:..debug/sched # echo NO_DELAY_DEQUEUE > features
rpi5:..debug/sched # /usr/bin/stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2149] setting to a 20 second run per stressor
stress-ng: info:  [2149] dispatching hogs: 4 pipe
stress-ng: metrc: [2149] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2149]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2149] pipe           32226171     20.00     17.93     61.98   1611264.75      403306.31
stress-ng: metrc: [2149] miscellaneous metrics:
stress-ng: metrc: [2149] pipe                 196.70 MB per sec pipe write rate (geometic mean of 4 instances)
stress-ng: info:  [2149] successful run completed in 20.01s
rpi5:..debug/sched # echo DELAY_DEQUEUE > features
rpi5:..debug/sched # /usr/bin/stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2210] setting to a 20 second run per stressor
stress-ng: info:  [2210] dispatching hogs: 4 pipe
stress-ng: metrc: [2210] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2210]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2210] pipe           32762333     20.00     18.08     61.86   1638067.42      409835.28
stress-ng: metrc: [2210] miscellaneous metrics:
stress-ng: metrc: [2210] pipe                 199.99 MB per sec pipe write rate (geometic mean of 4 instances)
stress-ng: info:  [2210] successful run completed in 20.01s