Re: [PATCH 8/9] PCI: Ignore BAR contents when firmware left decoding disabled

[Bjorn Helgaas]

[+cc kvm list]

On Wed, Mar 19, 2014 at 7:32 PM, Ming Lei <tom.leiming@xxxxxxxxx> wrote:
> On Thu, Mar 20, 2014 at 12:45 AM, Bjorn Helgaas <bhelgaas@xxxxxxxxxx> wrote:
>> On Tue, Mar 18, 2014 at 10:52 PM, Ming Lei <tom.leiming@xxxxxxxxx> wrote:
>>> Hi,
>>>
>>> Looks Sasha fixed the problem in lkvm tool[1].
>>>
>>> Sasha, looks we both saw the problem, but from technical
>>> view, I am wondering if the fix is correct, because PCI spec.
>>> requires that the IO/MMIO bits in COMMAND register should
>>> be cleared after reset, maybe there are some potential problem
>>> in lkvm pci emulation.
>>
>> I think I'm going to revert this patch ([2], "Ignore BAR contents when
>> firmware left decoding disabled").  The main reason for that patch was
>> to try for a consistent way of figuring out whether BARs are valid
>> that we could use on all architectures, but I think we can do it in a
>> better way.
>>
>> That said, this kvm change should not be necessary.  We *should* be
>> able to take any PCI device and initialize it from power-on state
>> without any dependencies on what the BIOS left in the BARs or the
>> command register.  As far as I can tell, the PCI core actually worked
>> fine in this case (we assigned valid addresses to the devices), but
>> something else blew up.  If I revert that patch, it will cover up
>> whatever this other bug is, but it would be much better to figure out
>> what it is and fix is.

I think I figured out what the problem is.  In virtio_pci__init(), we
allocate some address space with pci_get_io_space_block(), save its
address in vpci->mmio_addr, and hook that address space up to
virtio_pci__io_mmio_callback with kvm__register_mmio().

But when we update the BAR value in pci__config_wr(), the address
space mapping is never updated.  I think this means that virtio-pci
can't tolerate its devices being moved by the OS.

In my opinion, this is a bug in linux-kvm.  We've managed to avoid
triggering this bug by preventing Linux from moving the BAR (either by
me reverting my patch, or by Sasha's linux-kvm change [1]).  But it's
not very robust to assume that the OS will never change the BAR, so
it's quite possible that you'll trip over this again in the future.

Bjorn

[1] 6478ce1416aa kvm tools: mark our PCI card as PIO and MMIO able
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/