Re: [RFC Patch 00/12] IXGBE: Add live migration support for SRIOV NIC

[Alexander Duyck]

On 10/25/2015 10:36 PM, Lan Tianyu wrote:
> On 2015å10æ24æ 02:36, Alexander Duyck wrote:
> > I was thinking about it and I am pretty sure the dummy write approach is
> > problematic at best.  Specifically the issue is that while you are
> > performing a dummy write you risk pulling in descriptors for data that
> > hasn't been dummy written to yet.  So when you resume and restore your
> > descriptors you will have once that may contain Rx descriptors
> > indicating they contain data when after the migration they don't.
> How about changing sequence? dummy writing Rx packet data fist and then
> its desc. This can ensure that RX data is migrated before its desc and
> prevent such case.

No.  I think you are missing the fact that there are 256 descriptors per 
page.  As such if you dirty just 1 you will be pulling in 255 more, of 
which you may or may not have pulled in the receive buffer for.

So for example if you have the descriptor ring size set to 256 then that 
means you are going to get whatever the descriptor ring has since you 
will be marking the entire ring dirty with every packet processed, 
however you cannot guarantee that you are going to get all of the 
receive buffers unless you go through and flush the entire ring prior to 
migrating.

This is why I have said you will need to do something to force the rings 
to be flushed such as initiating a PM suspend prior to migrating.  You 
need to do something to stop the DMA and flush the remaining Rx buffers 
if you want to have any hope of being able to migrate the Rx in a 
consistent state.  Beyond that the only other thing you have to worry 
about are the Rx buffers that have already been handed off to the 
stack.  However those should be handled if you do a suspend and somehow 
flag pages as dirty when they are unmapped from the DMA.

- Alex
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/