Re: some trouble when using vrf

From: David Ahern
Date: Tue Apr 16 2019 - 11:20:56 EST


On 4/16/19 5:26 AM, linmiaohe wrote:
> Hi all:
> I meet some trouble when using vrf, it's very nice of you if you
> can help me fix it.
>
> 1.If I bind a vrf address before I set SO_BINDTODEVICE opts to bind an
> enslaved device, "Cannot assign requested address" will occurs. It's
> because only after we set SO_BINDTODEVICE opts to bind an enslaved device,
> then we can lookup the route in the specified l3mdev domain.
> I think we couldn't limit the order of bind ip addr and SO_BINDTODEVICE opts
> otherwise many vrf unaware apps many need to change their code.
> I can't find a convenient way to fix it.Here is an example:
>
> ...
> bind(sock_fd, (struct sockaddr *)&addr_serv, sizeof(addr_serv));
> ...
> ret = setsockopt(sock_fd, SOL_SOCKET, SO_BINDTODEVICE, bind_dev, strlen(bind_dev)+1);
> ...
> This code snipet doesn't work if the ip address of addr_serv is not in default vrf. And
> "Cannot assign requested address" will occurs.

Applications must bind to the device first and then bind to the local
address. As I recall this applies regardless of whether it is a VRF or
other.


>
> 2.When I run a udp server in default vrf and set net.ipv4.udp_l3mdev_accept=1,
> it's supposed to work accorss VRFs.
> But it occurs when udp server received a packet, it sends back with the source
> ip address from default vrf which is supposed to from the specified l3mdev domain.
> And the connection between server and client is broken. It's because udp server
> run in default vrf and doesn't bind the enslaved device, so only the route table in
> default vrf is avaliable.

UDP applications can use cmsg with IP_PKTINFO to send the reply out the
same device as the request was received. An app can also track the
relationship of the enslaved device to a VRF to use a VRF based lookup
to send a response. Choices.


>
> For example,I run a udp server in default vrf and set net.ipv4.udp_l3mdev_accept=1:
> [root@localhost vrf]# sysctl -a | grep l3mdev
> net.ipv4.raw_l3mdev_accept = 1
> net.ipv4.tcp_l3mdev_accept = 1
> net.ipv4.udp_l3mdev_accept = 1
>
> Then connect the server and send a udp packet to the specified vrf. Here is tcpdump output:
> 17:28:54.925417 IP 10.0.0.2.43003 > 10.0.0.1.irdmi: UDP, length 17
> 17:28:54.925953 IP 9.85.153.236.irdmi > 10.0.0.2.43003: UDP, length 17
> ps. 10.0.0.2 is client ip, 10.0.0.1 is specified vrf ip, and 9.85.153.236 is default vrf ip.
> 10.0.0.1 and 9.85.153.236 are in the same host.
>
> Thanks a lot if you can help me fix these. Have a nice day.
>
>
>