Re: [External] Re: next/master bisection: baseline.login on qemu_i386

[Chengming Zhou]

On 2022/3/10 12:44 上午, Mark Brown wrote:
> On Wed, Mar 09, 2022 at 05:12:07AM -0800, KernelCI bot wrote:
> 
> The KernelCI bisection bot found that commit dc6e0818bc9a0336d9a
> ("sched/cpuacct: Optimize away RCU read lock") has been causing boot
> failures for i386 qemu systems running a defconfig with debug options
> added in -next.  The system comes up to userspace but the shell is
> unresponsive, most other architectures and configs (including i386
> without debug options) seem fine.
> 
> I've left the full bisection report including links to logs below, you
> can also see a full overview of the baseline tests for -next today at:
> 
>     https://linux.kernelci.org/test/job/next/branch/master/kernel/next-20220309/plan/baseline/

Hi Mark, thanks for the bisection report. I've send a fix patch[1] for review.

[1] https://lore.kernel.org/lkml/20220305034103.57123-1-zhouchengming@xxxxxxxxxxxxx/

> 
>> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
>> * This automated bisection report was sent to you on the basis  *
>> * that you may be involved with the breaking commit it has      *
>> * found.  No manual investigation has been done to verify it,   *
>> * and the root cause of the problem may be somewhere else.      *
>> *                                                               *
>> * If you do send a fix, please include this trailer:            *
>> *   Reported-by: "kernelci.org bot" <bot@xxxxxxxxxxxx>          *
>> *                                                               *
>> * Hope this helps!                                              *
>> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
>>
>> next/master bisection: baseline.login on qemu_i386
>>
>> Summary:
>>   Start:      cb153b68ff91 Add linux-next specific files for 20220308
>>   Plain log:  https://storage.kernelci.org/next/master/next-20220308/i386/i386_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_i386.txt
>>   HTML log:   https://storage.kernelci.org/next/master/next-20220308/i386/i386_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_i386.html
>>   Result:     dc6e0818bc9a sched/cpuacct: Optimize away RCU read lock
>>
>> Checks:
>>   revert:     PASS
>>   verify:     PASS
>>
>> Parameters:
>>   Tree:       next
>>   URL:        https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
>>   Branch:     master
>>   Target:     qemu_i386
>>   CPU arch:   i386
>>   Lab:        lab-baylibre
>>   Compiler:   gcc-10
>>   Config:     i386_defconfig+debug
>>   Test case:  baseline.login
>>
>> Breaking commit found:
>>
>> -------------------------------------------------------------------------------
>> commit dc6e0818bc9a0336d9accf3ea35d146d72aa7a18
>> Author: Chengming Zhou <zhouchengming@xxxxxxxxxxxxx>
>> Date:   Sun Feb 20 13:14:25 2022 +0800
>>
>>     sched/cpuacct: Optimize away RCU read lock
>>     
>>     Since cpuacct_charge() is called from the scheduler update_curr(),
>>     we must already have rq lock held, then the RCU read lock can
>>     be optimized away.
>>     
>>     And do the same thing in it's wrapper cgroup_account_cputime(),
>>     but we can't use lockdep_assert_rq_held() there, which defined
>>     in kernel/sched/sched.h.
>>     
>>     Suggested-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
>>     Signed-off-by: Chengming Zhou <zhouchengming@xxxxxxxxxxxxx>
>>     Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
>>     Link: https://lore.kernel.org/r/20220220051426.5274-2-zhouchengming@xxxxxxxxxxxxx
>>
>> diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
>> index 75c151413fda..9a109c6ac0e0 100644
>> --- a/include/linux/cgroup.h
>> +++ b/include/linux/cgroup.h
>> @@ -791,11 +791,9 @@ static inline void cgroup_account_cputime(struct task_struct *task,
>>  
>>  	cpuacct_charge(task, delta_exec);
>>  
>> -	rcu_read_lock();
>>  	cgrp = task_dfl_cgroup(task);
>>  	if (cgroup_parent(cgrp))
>>  		__cgroup_account_cputime(cgrp, delta_exec);
>> -	rcu_read_unlock();
>>  }
>>  
>>  static inline void cgroup_account_cputime_field(struct task_struct *task,
>> diff --git a/kernel/sched/cpuacct.c b/kernel/sched/cpuacct.c
>> index 307800586ac8..f79f88456d72 100644
>> --- a/kernel/sched/cpuacct.c
>> +++ b/kernel/sched/cpuacct.c
>> @@ -337,12 +337,10 @@ void cpuacct_charge(struct task_struct *tsk, u64 cputime)
>>  	unsigned int cpu = task_cpu(tsk);
>>  	struct cpuacct *ca;
>>  
>> -	rcu_read_lock();
>> +	lockdep_assert_rq_held(cpu_rq(cpu));
>>  
>>  	for (ca = task_ca(tsk); ca; ca = parent_ca(ca))
>>  		*per_cpu_ptr(ca->cpuusage, cpu) += cputime;
>> -
>> -	rcu_read_unlock();
>>  }
>>  
>>  /*
>> -------------------------------------------------------------------------------
>>
>>
>> Git bisection log:
>>
>> -------------------------------------------------------------------------------
>> git bisect start
>> # good: [ea4424be16887a37735d6550cfd0611528dbe5d9] Merge tag 'mtd/fixes-for-5.17-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux
>> git bisect good ea4424be16887a37735d6550cfd0611528dbe5d9
>> # bad: [cb153b68ff91cbc434f3de70ac549e110543e1bb] Add linux-next specific files for 20220308
>> git bisect bad cb153b68ff91cbc434f3de70ac549e110543e1bb
>> # good: [1ce7aac49a7b73abbd691c6e6a1577a449d90bad] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
>> git bisect good 1ce7aac49a7b73abbd691c6e6a1577a449d90bad
>> # bad: [08688e100b1b07ce178c1d3c6b9983e00cd85413] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
>> git bisect bad 08688e100b1b07ce178c1d3c6b9983e00cd85413
>> # good: [3255b3a262be263864fcc5d5a239aebd1d0b2bb4] Merge branch 'drm-next' of https://gitlab.freedesktop.org/agd5f/linux
>> git bisect good 3255b3a262be263864fcc5d5a239aebd1d0b2bb4
>> # good: [983377cbb2ba7038a34b68f43788e78b34cb01d6] Merge branch 'pcmcia-next' of git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux.git
>> git bisect good 983377cbb2ba7038a34b68f43788e78b34cb01d6
>> # good: [71e306ad46cd73f05b9ba151161b4a3c1bd25e5d] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git
>> git bisect good 71e306ad46cd73f05b9ba151161b4a3c1bd25e5d
>> # bad: [a66320f041601309f1885bcb52e27e0c2f72079b] Merge branch into tip/master: 'core/core'
>> git bisect bad a66320f041601309f1885bcb52e27e0c2f72079b
>> # good: [9eb0c9821cf7efd5d18e2707caa01691c6d4fbf6] Merge branch into tip/master: 'x86/build'
>> git bisect good 9eb0c9821cf7efd5d18e2707caa01691c6d4fbf6
>> # bad: [49bef33e4b87b743495627a529029156c6e09530] sched/rt: Plug rt_mutex_setprio() vs push_rt_task() race
>> git bisect bad 49bef33e4b87b743495627a529029156c6e09530
>> # good: [2cfb7a1b031b0e816af7a6ee0c6ab83b0acdf05a] sched/fair: Improve consistency of allowed NUMA balance calculations
>> git bisect good 2cfb7a1b031b0e816af7a6ee0c6ab83b0acdf05a
>> # good: [65e53f869e9f92a23593c66214b88e54fb190a13] sched/isolation: Fix housekeeping_mask memory leak
>> git bisect good 65e53f869e9f92a23593c66214b88e54fb190a13
>> # good: [99cf983cc8bca4adb461b519664c939a565cfd4d] sched/preempt: Add PREEMPT_DYNAMIC using static keys
>> git bisect good 99cf983cc8bca4adb461b519664c939a565cfd4d
>> # good: [6255b48aebfd4dff375e97fc8b075a235848db0b] Merge tag 'v5.17-rc5' into sched/core, to resolve conflicts
>> git bisect good 6255b48aebfd4dff375e97fc8b075a235848db0b
>> # bad: [dc6e0818bc9a0336d9accf3ea35d146d72aa7a18] sched/cpuacct: Optimize away RCU read lock
>> git bisect bad dc6e0818bc9a0336d9accf3ea35d146d72aa7a18
>> # good: [248cc9993d1cc12b8e9ed716cc3fc09f6c3517dd] sched/cpuacct: Fix charge percpu cpuusage
>> git bisect good 248cc9993d1cc12b8e9ed716cc3fc09f6c3517dd
>> # first bad commit: [dc6e0818bc9a0336d9accf3ea35d146d72aa7a18] sched/cpuacct: Optimize away RCU read lock
>> -------------------------------------------------------------------------------