git checkout f722406faae2d073cc1d01063d1123c35425939e Previous HEAD position was f284b77... cgroup: use percpu refcnt for cgroup_subsys_states HEAD is now at f722406... Linux 3.10-rc1 ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:f722406faae2d073cc1d01063d1123c35425939e:bisect-linux 2013-06-13-09:06:37 f722406faae2d073cc1d01063d1123c35425939e reuse /kernel/x86_64-randconfig-a02-0612/f722406faae2d073cc1d01063d1123c35425939e/vmlinuz-3.10.0-rc1 2013-06-13-09:06:37 detecting boot state ..... 1 2 4 6 7 10 11 13 18.. 21.. 25. 26 27. 30 32.. 33. 34. 36 41 47 50 53 56 58 60 SUCCESS bisect: good commit f722406faae2d073cc1d01063d1123c35425939e git bisect start f284b77333b1320491f138b76f11006bbc962569 f722406faae2d073cc1d01063d1123c35425939e -- Previous HEAD position was f722406... Linux 3.10-rc1 Switched to branch 'master' Your branch is behind 'origin/master' by 3048 commits, and can be fast-forwarded. Bisecting: 584 revisions left to test after this (roughly 9 steps) [c7153d0643dd161df9e7a2fcb952418055aa5fca] Merge tag 'driver-core-3.10-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core git bisect run /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:c7153d0643dd161df9e7a2fcb952418055aa5fca:bisect-linux 2013-06-13-09:27:05 c7153d0643dd161df9e7a2fcb952418055aa5fca compiling 182 real 1635 user 141 sys 975.49% cpu x86_64-randconfig-a02-0612 2013-06-13-09:30:23 detecting boot state 3.10.0-rc2-00163-gc7153d0... 5 7 13 20 21 25 26 29 32 36 43. 50 54 58 60 SUCCESS Bisecting: 291 revisions left to test after this (roughly 8 steps) [088d812fe97a4cc8edaed69f91511ea063b14cef] Merge tag 'for-linus-v3.10-rc3' of git://oss.sgi.com/xfs/xfs running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:088d812fe97a4cc8edaed69f91511ea063b14cef:bisect-linux 2013-06-13-09:39:59 088d812fe97a4cc8edaed69f91511ea063b14cef compiling 178 real 1661 user 146 sys 1014.94% cpu x86_64-randconfig-a02-0612 2013-06-13-09:43:11 detecting boot state 3.10.0-rc2-00456-g088d812... 4 6 7 9 12. 16 19 29 33 36 38 43 53 54 57.. 58 60 SUCCESS Bisecting: 145 revisions left to test after this (roughly 7 steps) [977b55cf988b86cdc929fef730a78aca98a8ebdb] Merge tag 'please-pull-aertracefix' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:977b55cf988b86cdc929fef730a78aca98a8ebdb:bisect-linux 2013-06-13-09:54:44 977b55cf988b86cdc929fef730a78aca98a8ebdb compiling 164 real 1636 user 142 sys 1079.38% cpu x86_64-randconfig-a02-0612 2013-06-13-09:57:45 detecting boot state 3.10.0-rc3-00126-g977b55c.... 3 9 10 11 14 15 24 27 28 29. 30 35 41 45 46 47 52 59.................... 60 SUCCESS Bisecting: 74 revisions left to test after this (roughly 6 steps) [aa3ae6de36c86994c3446bdbc6b69ece9193732a] Merge branch 'merge' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:aa3ae6de36c86994c3446bdbc6b69ece9193732a:bisect-linux 2013-06-13-10:19:50 aa3ae6de36c86994c3446bdbc6b69ece9193732a compiling 175 real 1638 user 142 sys 1014.93% cpu x86_64-randconfig-a02-0612 2013-06-13-10:23:01 detecting boot state 3.10.0-rc3-00197-gaa3ae6d... 1 3 4 13 19 24 25 28 31 35 38 39 40 46 52 57 59 60 SUCCESS Bisecting: 39 revisions left to test after this (roughly 5 steps) [0ab60871b451b857206d7afc0d3033be866f76b4] Merge tag 'jfs-3.10-rc5' of git://github.com/kleikamp/linux-shaggy running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:0ab60871b451b857206d7afc0d3033be866f76b4:bisect-linux 2013-06-13-10:33:35 0ab60871b451b857206d7afc0d3033be866f76b4 compiling 311 real 1639 user 142 sys 572.08% cpu x86_64-randconfig-a02-0612 2013-06-13-10:42:39 detecting boot state 3.10.0-rc4-00024-g0ab6087.... 1 3 4 5.. 10 17. 18 22 23 24 32 34. 37 38. 42 45 46 47 52 53 56. 57. 58 59.. 60 SUCCESS Bisecting: 24 revisions left to test after this (roughly 4 steps) [de3c0749e2c1960afcc433fc5da136b85c8bd896] percpu-refcount: implement percpu_tryget() along with percpu_ref_kill_and_confirm() running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:de3c0749e2c1960afcc433fc5da136b85c8bd896:bisect-linux 2013-06-13-11:01:45 de3c0749e2c1960afcc433fc5da136b85c8bd896 compiling 180 real 1663 user 142 sys 1001.75% cpu x86_64-randconfig-a02-0612 2013-06-13-11:04:58 detecting boot state 3.10.0-rc4-00039-gde3c074........ 1... 2. 3 4.. 5 6.. 8. 9 10 11 12 13 15 21 24 29 32 34 43 46 47. 48 50 52 54 56 59............. 60 SUCCESS Bisecting: 12 revisions left to test after this (roughly 4 steps) [cc5943a7816ba6c00639837a62131386619548dc] cgroup: mark "notify_on_release" and "release_agent" cgroup files insane running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:cc5943a7816ba6c00639837a62131386619548dc:bisect-linux 2013-06-13-11:34:35 cc5943a7816ba6c00639837a62131386619548dc compiling 189 real 1653 user 156 sys 953.09% cpu x86_64-randconfig-a02-0612 2013-06-13-11:37:56 detecting boot state 3.10.0-rc1-00014-gcc5943a... 1 4.. 7 8 9 12 13 14 16 17 19 20.. 21 23 25 26 28 31 33 34 37 41.... 42. 45 46... 48 49.. 51. 52. 55. 56 58. 60 SUCCESS Bisecting: 6 revisions left to test after this (roughly 3 steps) [b5f1b7123762411e6a432f8236304e4dcaf5b4d8] cgroup: use kzalloc() and list_del_init() running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:b5f1b7123762411e6a432f8236304e4dcaf5b4d8:bisect-linux 2013-06-13-12:05:02 b5f1b7123762411e6a432f8236304e4dcaf5b4d8 compiling 178 real 1654 user 141 sys 1008.41% cpu x86_64-randconfig-a02-0612 2013-06-13-12:08:14 detecting boot state 3.10.0-rc4-00057-gb5f1b71... 4 6 7 9.. 10 11. 14. 15. 16 20 23.. 25 26 28 29 34. 38 45 51 52. 53 57 59 60 SUCCESS Bisecting: 3 revisions left to test after this (roughly 2 steps) [919e0c4a614fce2fdf8a57e5500841e18bb523e6] cgroup: drop unnecessary RCU dancing from __put_css_set() running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:919e0c4a614fce2fdf8a57e5500841e18bb523e6:bisect-linux 2013-06-13-12:26:20 919e0c4a614fce2fdf8a57e5500841e18bb523e6 compiling 177 real 1654 user 137 sys 1008.07% cpu x86_64-randconfig-a02-0612 2013-06-13-12:29:34 detecting boot state 3.10.0-rc4-00060-g919e0c4... 1 6 16 27 32 37. 49 57 60 SUCCESS Bisecting: 1 revision left to test after this (roughly 1 step) [fbcc713b2b109bd180688b65c3967d5a1cd46010] cgroup: reorder the operations in cgroup_destroy_locked() running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:fbcc713b2b109bd180688b65c3967d5a1cd46010:bisect-linux 2013-06-13-12:36:09 fbcc713b2b109bd180688b65c3967d5a1cd46010 compiling 163 real 1654 user 135 sys 1095.37% cpu x86_64-randconfig-a02-0612 2013-06-13-12:39:07 detecting boot state 3.10.0-rc4-00062-gfbcc713.... 1.. 2 3 9 12 14. 15 19 22.. 24 38 41 42 45 48 51 56 60 SUCCESS Bisecting: 0 revisions left to test after this (roughly 0 steps) [903c63033fff6a0f2f60b7c71831fe3b25ea9abc] cgroup: split cgroup destruction into two steps running /c/kernel-tests/bisect-test-boot-failure.sh /home/wfg/net/obj-bisect ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:903c63033fff6a0f2f60b7c71831fe3b25ea9abc:bisect-linux 2013-06-13-12:52:41 903c63033fff6a0f2f60b7c71831fe3b25ea9abc compiling 166 real 1646 user 142 sys 1072.10% cpu x86_64-randconfig-a02-0612 2013-06-13-12:55:42 detecting boot state 3.10.0-rc4-00063-g903c630... 6 9 13 24 26 30 40 50 57 59.............................................................................................................................................................................................................................................................................................................add_to_run_queue 1 ... 60 SUCCESS f284b77333b1320491f138b76f11006bbc962569 is the first bad commit commit f284b77333b1320491f138b76f11006bbc962569 Author: Tejun Heo Date: Wed Jun 12 13:50:58 2013 -0700 cgroup: use percpu refcnt for cgroup_subsys_states A css (cgroup_subsys_state) is how each cgroup is represented to a controller. As such, it can be used in hot paths across the various subsystems different controllers are associated with. One of the common operations is reference counting, which up until now has been implemented using a global atomic counter and can have significant adverse impact on scalability. For example, css refcnt can be gotten and put multiple times by blkcg for each IO request. For highops configurations which try to do as much per-cpu as possible, the global frequent refcnting can be very expensive. In general, given the various hugely diverse paths css's end up being used from, we need to make it cheap and highly scalable. In its usage, css refcnting isn't very different from module refcnting. This patch converts css refcnting to use the recently added percpu_ref. css_get/tryget/put() directly maps to the matching percpu_ref operations and the deactivation logic is no longer necessary as percpu_ref already has refcnt killing. The only complication is that as the refcnt is per-cpu, percpu_ref_kill() in itself doesn't ensure that further tryget operations will fail, which we need to guarantee before invoking ->css_offline()'s. This is resolved collecting kill confirmation using percpu_ref_kill_and_confirm() and initiating the offline phase of destruction after all css refcnt's are confirmed to be seen as killed on all CPUs. The previous patches already splitted destruction into two phases, so percpu_ref_kill_and_confirm() can be hooked up easily. This patch removes css_refcnt() which is used for rcu dereference sanity check in css_id(). While we can add a percpu refcnt API to ask the same question, css_id() itself is scheduled to be removed fairly soon, so let's not bother with it. Just drop the sanity check and use rcu_dereference_raw() instead. Signed-off-by: Tejun Heo Cc: Michal Hocko Cc: Mike Snitzer Cc: Vivek Goyal Cc: "Alasdair G. Kergon" Cc: Jens Axboe Cc: Mikulas Patocka Cc: Glauber Costa Cc: Kent Overstreet :040000 040000 f82ce639ba14ea710cc1db2c21f031fed7604709 f44626bd17a0d5c101fd7863a3a594be1145827c M include :040000 040000 e8935333a38f95762be947d94329ab4abc7e7af9 1243f529172759c11c2faef08755c82de7d25b56 M kernel bisect run success ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:903c63033fff6a0f2f60b7c71831fe3b25ea9abc:bisect-linux 2013-06-13-15:34:58 903c63033fff6a0f2f60b7c71831fe3b25ea9abc reuse /kernel/x86_64-randconfig-a02-0612/903c63033fff6a0f2f60b7c71831fe3b25ea9abc/vmlinuz-3.10.0-rc4-00063-g903c630 2013-06-13-15:34:58 detecting boot state ... 61. 62 65. 73. 74 75. 80 81. 82 83 85 89. 91 94 96 98 100 106 108 113. 117. 118 119 122. 126 128.............................................................................................................................................................................................................................................................................................................add_to_run_queue 52 .... 129. 130.. 131......... 132... 133........ 134.. 136. 137 140 141 144. 146 147. 148..... 149 152 156 158 159. 160.... 161 162.. 163... 164.. 165.... 166... 167.. 168 170. 172. 174 175 178 179 180 SUCCESS ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:f284b77333b1320491f138b76f11006bbc962569:bisect-linux TEST FAILURE /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-bay-28446-20130613090741-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-lkp-nex04-41895-20130613035323-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-lkp-nex04-43673-20130613035342-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-lkp-nex04-62379-20130613035349-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-lkp-nex04-64111-20130613035331-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-vp-25931-20130613050416-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-vp-28116-20130613050426-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-vp-29406-20130613050425-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-vp-30144-20130613090800-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-waimea-28190-20130613090858-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-waimea-28704-20130613050409-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-waimea-3709-20130613090902-3.10.0-rc4-00064-gf284b773-57 /kernel/x86_64-randconfig-a02-0612/f284b77333b1320491f138b76f11006bbc962569/dmesg-kvm-waimea-3785-20130613050415-3.10.0-rc4-00064-gf284b773-57 [detached HEAD a2c764c] Revert "cgroup: use percpu refcnt for cgroup_subsys_states" 2 files changed, 70 insertions(+), 91 deletions(-) ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:a2c764c7dbc632ab6ac48b2d833f44dce26ba1ed:bisect-linux 2013-06-13-19:12:12 a2c764c7dbc632ab6ac48b2d833f44dce26ba1ed compiling 2013-06-13-19:16:15 detecting boot state 3.10.0-rc4-00065-ga2c764c........ 1. 3.... 5. 7......... 8. 11...... 13 14.. 16.. 17....... 18............... 19 20.. 21......... 23. 24.. 25.................... 26............ 27...... 28.. 29. 30.. 31.............. 32......... 33............. 34....... 35 36........... 37. 38..... 39 40... 41 43. 44............ 45... 46. 48...................... 49.. 50. 51. 52............................. 53..... 54.................... 55.. 57....... 59...... 61. 62.. 63... 64 65. 66... 67 68 70. 71. 73. 74 75 80 85 89 90. 95 98 102. 105 115 117 122 123 128 133 136 140 141 143 144 146 149 150 151 152. 153.. 156 157. 159 160.. 163.. 164 165. 166 169.. 173 174 176.. 180 SUCCESS ========= upstream ========= Fetching linus ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:26e04462c8b78d079d3231396ec72d58a14f114b:bisect-linux 2013-06-13-22:43:50 26e04462c8b78d079d3231396ec72d58a14f114b reuse /kernel/x86_64-randconfig-a02-0612/26e04462c8b78d079d3231396ec72d58a14f114b/vmlinuz-3.10.0-rc5-00167-g26e0446 2013-06-13-22:43:50 detecting boot state .... 10... 11..... 12 13 14. 15 17.. 18 21. 22 26 27 29. 31 34 35. 36 38 39. 40.. 41. 43. 44. 45.. 47 48 49 51.. 52..... 53... 54 56. 57 58. 61 62. 64. 66............. 67 68.............. 69 70 72 73... 74. 75 76.... 77 78. 79... 80 81..... 82 83. 84 86. 87.. 90. 93 95 96 98 108....... 126 127. 128.... 129 130 132.. 133. 134 135 137 139 142. 143 144 146 148.. 150. 151 152 153 154.. 158 159.. 161 163. 164 166 167 169 171 172.. 174 176 177 178... 180 SUCCESS ========= linux-next ========= Fetching next ls -a /kernel-tests/run-queue/kvm/x86_64-randconfig-a02-0612/cgroup:review-css-percpu-ref:c04efed734409f5a44715b54a6ca1b54b0ccf215:bisect-linux 2013-06-14-00:32:53 c04efed734409f5a44715b54a6ca1b54b0ccf215 compiling 2013-06-14-00:37:29 detecting boot state 3.10.0-rc4-next-20130607-05001-gc04efed...... 1.. 2.. 4.... 5 6.... 7 8.......... 11.. 12 13 16............. 17.. 18... 20... 21... 23. 24...... 27....... 28.. 30..... 31. 32. 33 34..... 35. 36 37........... 38 39 40. 41.. 42.. 44.... 45. 46.... 47...... 48. 49.. 50 51... 52. 53 54... 55.. 56............ 57. 58 60.. 62. 63....... 64... 65... 66.. 67.... 68.. 69. 70.. 71..... 72 73........... 74. 75 76. 77...... 78.... 80 82......... 83.. 84. 85... 86 87.. 88............ 89............ 91.... 92..... 93. 94. 96......... 97 98... 99. 101 102 103 105 106 108. 109 110 112. 113 114. 118 120... 122.. 125 126. 129 131 132 135 136 140 143 149 155 157 159 161 162 164 165 169 172 175. 177 179 180 SUCCESS