Multicast MAC when sending via gateway

From: Nikita Rastegaev
Date: Mon Aug 09 2010 - 08:43:49 EST


I observe some strange behavior when sending multicast datagrams
via gateway. It appears that destination MAC in UDP packets differs
when using connected and connectionless socket. In case of
connectionless socket destination MAC field of UDP header is filled
with MAC address of gateway, but in case we use connect()
with multicast address, destination MAC is set to multicast MAC.
I don't know, which behavior is right, but different behavior seems
strange.

I run the attached test on thorin1 machine and use thorin2 as a gateway.
thorin1 is running 2.6.26-2-686 kernel, thorin2 is 2.6.26-2-amd64.
I also attach ifconfig and route output for this machines and tcpdump
output on thorin2 when running test. There we can see the following:

16:02:30.567909 00:27:0e:09:01:2a > 00:27:0e:09:02:2a, ethertype IPv4
(0x0800), length 58: 10.54.10.1.40074 > 237.92.130.210.51721: UDP, length
16
16:02:30.567984 00:27:0e:09:01:2a > 01:00:5e:5c:82:d2, ethertype IPv4
(0x0800), length 74: 10.54.10.1.43360 > 237.92.130.210.51721: UDP, length
32

Is it a bug or is this kind of behavior expected?

I wish to be personally CC'ed the answers/comments posted to the list in
response to this posting.

Nikita Rastegaev <Nikita.Rastegaev@xxxxxxxxxxxx>eth0 Link encap:Ethernet HWaddr 00:27:0e:09:01:1a
inet addr:192.168.38.121 Bcast:192.168.38.255 Mask:255.255.255.0
inet6 addr: fe80::227:eff:fe09:11a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3771108 errors:0 dropped:0 overruns:0 frame:0
TX packets:1108184 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2000489426 (1.8 GiB) TX bytes:152178002 (145.1 MiB)

eth1 Link encap:Ethernet HWaddr 00:27:0e:09:01:2a
inet addr:10.54.10.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::227:eff:fe09:12a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:529053 errors:0 dropped:0 overruns:0 frame:0
TX packets:571799 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:27234801 (25.9 MiB) TX bytes:583704428 (556.6 MiB)

eth2 Link encap:Ethernet HWaddr 00:27:0e:09:01:3a
inet addr:10.54.11.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::227:eff:fe09:13a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:42 errors:0 dropped:0 overruns:0 frame:0
TX packets:1448 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5352 (5.2 KiB) TX bytes:113076 (110.4 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:38022106 errors:0 dropped:0 overruns:0 frame:0
TX packets:38022106 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:728488688 (694.7 MiB) TX bytes:728488688 (694.7 MiB)

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
237.92.130.210 10.54.10.2 255.255.255.255 UGH 0 0 0 eth1
192.168.38.0 * 255.255.255.0 U 0 0 0 eth0
10.0.0.0 * 255.0.0.0 U 0 0 0 eth1
10.0.0.0 * 255.0.0.0 U 0 0 0 eth2
default umbar.oktetlabs 0.0.0.0 UG 0 0 0 eth0
eth0 Link encap:Ethernet HWaddr 00:27:0e:09:02:1a
inet addr:192.168.38.122 Bcast:192.168.38.255 Mask:255.255.255.0
inet6 addr: fe80::227:eff:fe09:21a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5126439 errors:0 dropped:0 overruns:0 frame:0
TX packets:2798221 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2289533079 (2.1 GiB) TX bytes:279425803 (266.4 MiB)

eth1 Link encap:Ethernet HWaddr 00:27:0e:09:02:2a
inet addr:10.54.10.2 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::227:eff:fe09:22a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:571635 errors:0 dropped:0 overruns:0 frame:0
TX packets:530295 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:583816604 (556.7 MiB) TX bytes:27333401 (26.0 MiB)

eth2 Link encap:Ethernet HWaddr 00:27:0e:09:02:3a
inet addr:10.54.12.1 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::227:eff:fe09:23a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:1413 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:110758 (108.1 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4373 errors:0 dropped:0 overruns:0 frame:0
TX packets:4373 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1320308 (1.2 MiB) TX bytes:1320308 (1.2 MiB)

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.38.0 * 255.255.255.0 U 0 0 0 eth0
10.0.0.0 * 255.0.0.0 U 0 0 0 eth1
10.0.0.0 * 255.0.0.0 U 0 0 0 eth2
default umbar.oktetlabs 0.0.0.0 UG 0 0 0 eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
16:02:25.567806 00:27:0e:09:01:2a > 00:27:0e:09:02:2a, ethertype IPv4 (0x0800), length 43: 10.54.10.1.40074 > 10.54.10.2.51721: UDP, length 1
16:02:25.567829 00:27:0e:09:02:2a > 00:27:0e:09:01:2a, ethertype IPv4 (0x0800), length 71: 10.54.10.2 > 10.54.10.1: ICMP 10.54.10.2 udp port 51721 unreachable, length 37
16:02:30.564271 00:27:0e:09:02:2a > 00:27:0e:09:01:2a, ethertype ARP (0x0806), length 42: arp who-has 10.54.10.1 tell 10.54.10.2
16:02:30.564736 00:27:0e:09:01:2a > 00:27:0e:09:02:2a, ethertype ARP (0x0806), length 42: arp who-has 10.54.10.2 tell 10.54.10.1
16:02:30.564741 00:27:0e:09:02:2a > 00:27:0e:09:01:2a, ethertype ARP (0x0806), length 42: arp reply 10.54.10.2 is-at 00:27:0e:09:02:2a
16:02:30.564755 00:27:0e:09:01:2a > 00:27:0e:09:02:2a, ethertype ARP (0x0806), length 42: arp reply 10.54.10.1 is-at 00:27:0e:09:01:2a
16:02:30.567909 00:27:0e:09:01:2a > 00:27:0e:09:02:2a, ethertype IPv4 (0x0800), length 58: 10.54.10.1.40074 > 237.92.130.210.51721: UDP, length 16
16:02:30.567984 00:27:0e:09:01:2a > 01:00:5e:5c:82:d2, ethertype IPv4 (0x0800), length 74: 10.54.10.1.43360 > 237.92.130.210.51721: UDP, length 32

/*
* Simple test to determine destination mac of multicast packets
* sent via gateway.
*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/select.h>
#include <netinet/in.h>
#include <arpa/inet.h>

#define TST_PORT 51721

int
main(int argc, const char **argv)
{
int fd1, fd2;
int rc;
struct sockaddr_in sa;
struct sockaddr_in gw;
char sendbuf[32];

/* Allocate multicast address */
memset(&sa, 0, sizeof(sa));
sa.sin_family = AF_INET;
sa.sin_port = htons(TST_PORT);
if (inet_pton(AF_INET, "237.92.130.210", &sa.sin_addr) < 0)
{
printf("Address resolving error\n");
exit(1);
}

/* Allocate gateway address */
memset(&gw, 0, sizeof(gw));
gw.sin_family = AF_INET;
gw.sin_port = htons(TST_PORT);
if (inet_pton(AF_INET, "10.54.10.2", &gw.sin_addr) < 0)
{
printf("Address resolving error\n");
exit(1);
}

/* Create sockets */
fd1 = socket(AF_INET, SOCK_DGRAM, 0);
fd2 = socket(AF_INET, SOCK_DGRAM, 0);
if (fd1 < 0 || fd2 < 0)
{
printf("socket error\n");
exit(1);
}

/* Send packet to gateway in order to get ARP reply */
rc = sendto(fd1, sendbuf, 1, 0, (struct sockaddr *)&gw, sizeof(gw));

if (rc < 0)
{
printf("sendto error on fd1 when sending to gw\n");
exit(1);
}

sleep(5);

/* Send some data from connectionless socket */
rc = sendto(fd1, sendbuf, 16, 0, (struct sockaddr *)&sa, sizeof(sa));

if (rc < 0)
{
printf("sendto error on fd1 when sending to sa\n");
exit(1);
}

/* Connect the second socket */
if (connect(fd2, (struct sockaddr *)&sa, sizeof(sa)) < 0)
{
printf("connect error\n");
exit(1);
}

/* Send some data from connected socket */
rc = send(fd2, sendbuf, 32, 0);

if (rc < 0)
{
printf("send error on fd2\n");
exit(1);
}

exit(0);
}