Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

[Christian Borntraeger]

On 11/21/2017 07:09 PM, Jens Axboe wrote:
> On 11/21/2017 10:27 AM, Jens Axboe wrote:
>> On 11/21/2017 03:14 AM, Christian Borntraeger wrote:
>>> Bisect points to
>>>
>>> 1b5a7455d345b223d3a4658a9e5fce985b7998c1 is the first bad commit
>>> commit 1b5a7455d345b223d3a4658a9e5fce985b7998c1
>>> Author: Christoph Hellwig <hch@xxxxxx>
>>> Date:   Mon Jun 26 12:20:57 2017 +0200
>>>
>>>     blk-mq: Create hctx for each present CPU
>>>     
>>>     commit 4b855ad37194f7bdbb200ce7a1c7051fecb56a08 upstream.
>>>     
>>>     Currently we only create hctx for online CPUs, which can lead to a lot
>>>     of churn due to frequent soft offline / online operations.  Instead
>>>     allocate one for each present CPU to avoid this and dramatically simplify
>>>     the code.
>>>     
>>>     Signed-off-by: Christoph Hellwig <hch@xxxxxx>
>>>     Reviewed-by: Jens Axboe <axboe@xxxxxxxxx>
>>>     Cc: Keith Busch <keith.busch@xxxxxxxxx>
>>>     Cc: linux-block@xxxxxxxxxxxxxxx
>>>     Cc: linux-nvme@xxxxxxxxxxxxxxxxxxx
>>>     Link: http://lkml.kernel.org/r/20170626102058.10200-3-hch@xxxxxx
>>>     Signed-off-by: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
>>>     Cc: Oleksandr Natalenko <oleksandr@xxxxxxxxxxxxxx>
>>>     Cc: Mike Galbraith <efault@xxxxxx>
>>>     Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
>>
>> I wonder if we're simply not getting the masks updated correctly. I'll
>> take a look.
> 
> Can't make it trigger here. We do init for each present CPU, which means
> that if I offline a few CPUs here and register a queue, those still show
> up as present (just offline) and get mapped accordingly.
> 
> From the looks of it, your setup is different. If the CPU doesn't show
> up as present and it gets hotplugged, then I can see how this condition
> would trigger. What environment are you running this in? We might have
> to re-introduce the cpu hotplug notifier, right now we just monitor
> for a dead cpu and handle that.

I am not doing a hot unplug and the replug, I use KVM and add a previously
not available CPU.

in libvirt/virsh speak:
  <vcpu placement='static' current='1'>4</vcpu>