Policy routing - overlapping subnets - broken arp

From: Pieter Smit
Date: Sun Dec 13 2009 - 02:01:53 EST

I am stuck setting up the following senario.

[A] Senario:
Linux gateway (T1) with 3 Ethernet cards eth0,eth1,eth2. <->eth1@T1 <->eth2@T1<->eth0:1@T1 (nat to eth1-plc1)<->eth0:2@T1 (nat to eth2-plc2)

The xp-pc need to be able to talk to both plc's through natted IP's.
e.g. xp(ping) [s192.168.3.155, d192.168.3.201]
-> T1-DnatPreRoute [s192.168.3.155, d10.0.0.1(eth1)]
-> T1-SnatPostRoute [s10.0.0.101, d10.0.0.1(eth1)]
<- and then reply from

[B] How
Using connection marks and FW marks connections are marked as they
come in on eth0 and the mark values are used to select the correct
routing table(101/102) containing eth1 or eth2 as the outgoing
interface to plc.
We also setup SNAT of the outgoing packets on eth1/eth2 to hide the
XP( behind the interface IP on eth1/eth2 as it leaves
for the PLC's
Using tcpdump we have been able to confirm that policy routing send
packets from XP through T1 to the correct interface eth1/eth2

[C] Problem
c1-we are only able to ping through to one of the PLC's. Looking at
the tcpdumps T1-Linux-gw does not respond to arp requests on eth2
interface for it's locally configured IP. (although it(T1) arped for out the correct interface and sent the icmp request out)

If we down eth1 then the pings work fine out eth2, and if we bring
eth1 up again, eth1 has the problem.
All the time we are able to verify with tcpdump that the packets (echo
requests) are leaving using the correct interface.

Here is some more info:
# uname -a
Linux m2 2.6.31-16-generic-pae #52-Ubuntu SMP Thu Dec 3 23:18:13 UTC
2009 i686 GNU/Linux
# iptables -V >> iptables v1.4.4
# ip -V >> ip utility, iproute2-ss090324
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html