Re: [PATCH] ena: Speed up initialization 90x by reducing poll delays

[Machulsky, Zorik]

ïOn 3/2/20, 3:54 PM, "Jakub Kicinski" <kuba@xxxxxxxxxx> wrote:

        
    
    On Mon, 2 Mar 2020 23:16:32 +0000 Machulsky, Zorik wrote:
    > On 2/28/20, 4:29 PM, "Josh Triplett" <josh@xxxxxxxxxxxxxxxx> wrote:
    >
    >     Before initializing completion queue interrupts, the ena driver uses
    >     polling to wait for responses on the admin command queue. The ena driver
    >     waits 5ms between polls, but the hardware has generally finished long
    >     before that. Reduce the poll time to 10us.
    >
    >     On a c5.12xlarge, this improves ena initialization time from 173.6ms to
    >     1.920ms, an improvement of more than 90x. This improves server boot time
    >     and time to network bringup.
    >
    > Thanks Josh,
    > We agree that polling rate should be increased, but prefer not to do
    > it aggressively and blindly. For example linear backoff approach
    > might be a better choice. Please let us re-work a little this patch
    > and bring it to review. Thanks!
    
    Up to Josh if this is fine with him, but in my experience "let us rework
    your patch behind the close doors" is not the response open source
    contributors are expecting.

Not sure I'm following what you mean by "behind the close door". Everything is open here.
I offered that ENA folks would take it further, because such change require  (in addition to 
Implementation change that we propose) a careful  testing with different platforms and 
instance types. Having said that, Josh, if you would like to take care of it, we will gladly help.
And thank you again for catching this!