Unfortunately not. This commit has a side effect that it in fact
disables the multiqueue macvtap transmission. Since all macvtap queues
will contend on a single qdisc lock.
They will only contend on a single qdisc lock if the lower device has
I think we are talking about 6acf54f1cf0a6747bac9fea26f34cfc5a9029523.
The qdisc or txq lock were macvlan device itself since dev_queue_xmit()
was called for macvlan device itself. So even if lower device has
multiple txqs, if you just create a one queue macvlan device, you will
get lock contention on macvlan device. And even if you explicitly
specifying the txq numbers ( though I don't believe most management
software will do this) when creating the macvlan/macvtap device, you
must also configure the XPS for macvlan to make sure it has the
possibility of using multiple transmit queues.
Perhaps defaulting the L2 forwarding devices to 1queue was a
mistake. But the same issue arises when running macvtap over a
non-multiqueue nic. Or even if you have a multiqueue device and create
many more macvtap queues than the lower device has queues.
Shouldn't the macvtap configuration take into account the lowest level
See commit 8ffab51b3dfc54876f145f15b351c41f3f703195 ("macvlan: lockless
tx path"). It allows the management to create a device without worrying
the underlying device.
How does using the L2 forwarding device change the
contention issues? Without the L2 forwarding LLTX is enabled but the
qdisc lock, etc is still acquired on the device below the macvlan.
That's the point. We need make sure the txq selection and qdisc lock
were done for the lower device not for the macvlan device itself. Then
macvlan can automatically benefit from the multi-queue capable lower
devices. But L2 forwarding needs to contend on the txq lock on macvlan
device itself, which is unnecessary and can complex the management.
The ixgbe driver as it is currently written can be configured for up to
4 queues by setting numtxqueues when the device is created. I assume
when creating macvtap queues the user needs to account for the number
of queues supported by the lower device.
We'd better not complicate the task of management, lockless tx path work
very well so we can just keep it. Btw, there's no way for the user to
know the maximum number of queues that L2 forwarding supports.
For L2 forwarding offload itself, more issues need to be addressed for
- ndo_dfwd_add_station() can only create queues per device at ndo_open,
but multiqueue macvtap allows user to create and destroy queues at their
will and at any time.
same argument as above, isn't this the same when running macvtap without
the l2 offloads over a real device? I expect you hit the same contention
points when running over a real device.
Not true and not only for contention.
Macvtap allows user to create or destroy a queue by simply open or close
to character device /dev/tapX. But currently, we do nothing when a new
queue was created or destroyed for L2 forwarding offload.
For contention, lockless tx path make the contention only happens for
the txq or qdisc for the lower device, but L2 forwarding offload make
contention also happen for the macvlan device itself.
- it looks that ixgbe has a upper limit of 4 queues per station, but
macvtap currently allows up to 16 queues per device.
The 4 limit was to simplify the code because the queue mapping in the
driver gets complicated if it is greater than 4. We can probably
increase this latter. But sorry reiterating how is this different than
a macvtap on a real device that supports a max of 4 queues?
Well, it maybe easy. I just point out possible issues we may meet currently.
So more works need to be done and unless those above 3 issues were
addressed, this patch is really needed to make sure macvtap works.
Agreed there is a lot more work here to improve things I'm just not
sure we need to disable this now. Also note its the l2 forwarding
should be disabled by default so a user would have to enable the
Even if it was disabled by default. We should not surprise the user who
want to enable it for macvtap.