Re: get/put_cpu() usage in block/blk-mq.c

[Jens Axboe]

On 05/09/2017 10:26 AM, Mike Galbraith wrote:
> On Tue, 2017-05-09 at 09:45 -0600, Jens Axboe wrote:
> 
>>>>>>> Is it from this_cpu_ptr() in blk_stat_add()?
>>>>>>
>>>>>> Yeah.
>>>>>
>>>>> So why is this complaining, doesn't rcu_read_lock() disable
>>>>> preemption?
>>>>
>>>> Ah, I guess it doesn't if PREEMPT_RCU is set. How about the
>>>> below?
>>>
>>> Should do it.  I was about to run LTP (where it turned up) again
>>> anyway, I'll add this.  No news is good news, what you should hear.
>>
>> Thanks, let me know, so I can add your tested-by (or whatever you
>> prefer).
> 
> Sure, but s/get_cpu_var/get_cpu_ptr first.

Doh.

diff --git a/block/blk-stat.c b/block/blk-stat.c
index 6c2f40940439..c52356d90fe3 100644
--- a/block/blk-stat.c
+++ b/block/blk-stat.c
@@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq)
 
 	rcu_read_lock();
 	list_for_each_entry_rcu(cb, &q->stats->callbacks, list) {
-		if (blk_stat_is_active(cb)) {
-			bucket = cb->bucket_fn(rq);
-			if (bucket < 0)
-				continue;
-			stat = &this_cpu_ptr(cb->cpu_stat)[bucket];
-			__blk_stat_add(stat, value);
-		}
+		if (!blk_stat_is_active(cb))
+			continue;
+
+		bucket = cb->bucket_fn(rq);
+		if (bucket < 0)
+			continue;
+
+		stat = &get_cpu_ptr(cb->cpu_stat)[bucket];
+		__blk_stat_add(stat, value);
+		put_cpu_ptr(cb->cpu_stat);
 	}
 	rcu_read_unlock();
 }

-- 
Jens Axboe