SOLVED: iptables MARK + ip rule fwmark on locally generated packets

From: Fredrik Ax
Date: Fri Jan 22 2010 - 07:11:38 EST


On Fri, Jan 22, 2010 at 11:17:09AM +0100, Fredrik Ax wrote:

> On Fri, Jan 22, 2010 at 11:09:02AM +0100, Fredrik Ax wrote:
>
> > Hi guys,
> >
> > I'm a pretty experienced Linux / network developer and administrator,
> > but I can't get my head around this one.
> >
> > The long story is that I have a box used as router/fw/proxy running
> > Debian Squeeze with a customized 2.6.32 x86_64 kernel having three
> > interfaces (eth2,eth3,eth4) on the same external subnet. One of the
> > interfaces is used for doing masquerading of other
> > subnets. Masquerading (not snat) is chosen because the interfaces are
> > on dhcp, and I don't want to have to rewrite the fw rules each time I
> > get a new addr ... already have enough with dhclient-hooks for fixing
> > the routing tables dns-updates, etc ;-) What I basically want to do is
> > make the proxy's request to go out the same ifc as the masqueraded
> > packets getting a src addr of s41.s42.s43.s44. Other locally generated
> > packets should get a src addr s21.s22.s23.s24.
> >
> > To accomplish this I'm using iptables to mark all, to port 80, locally
> > generated tcp packets:
> >
> > % iptables -t mangle -vnL OUTPUT
> > Chain OUTPUT (policy ACCEPT 3234 packets, 2254K bytes)
> > pkts bytes target prot opt in out source destination
> > 1114 181K MARK tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 MARK set 0x4
> >
> > I have verified that the iptables rule marks them fine enough.
> >
> > Then the ip rule with prio 99 below should then catch them and route
> > according to table eth4 below. That rule however does, for some reason
> > not match those packets, instead they are routed according to table
> > eth2 below (prio 200 rule), getting src addr s21.s22.s23.s24. If I
> > disable that rule they are routed according the the prio 300 rule
> > (getting src addr s31.s32.s33.s34).
> >
> > prompt% ip rule
> > 0: from all lookup local
> > 1: from all lookup main
> > 99: from all fwmark 0x4 lookup eth4
> > 100: from 10.116.254.0/26 lookup eth4
> > 100: from 10.116.255.34 lookup eth3
> > 100: from 10.116.255.64/26 lookup eth4
> > 200: from all lookup eth2
> > 300: from all lookup eth3
> > 400: from all lookup eth4
> > 32767: from all lookup default
> >
> > prompt% ip route show table eth2
> > broadcast b1.b2.b3.b4 dev eth2 scope link src s21.s22.s23.s24
> > broadcast n1.n2.n3.n4 dev eth2 scope link src s21.s22.s23.s24
> > n1.n2.n3.n4/m dev eth2 scope link src s21.s22.s23.s24
> > default via g1.g2.g3.g4 dev eth2 src s21.s22.s23.s24
> >
> > prompt% ip route show table eth3
> > broadcast b1.b2.b3.b4 dev eth3 scope link src s31.s32.s33.s34
> > broadcast n1.n2.n3.n4 dev eth3 scope link src s31.s32.s33.s34
> > n1.n2.n3.n4/m dev eth3 scope link src s31.s32.s33.s34
> > default via g1.g2.g3.g4 dev eth3 src s31.s32.s33.s34
> >
> > prompt% ip route show table eth4
> > broadcast b1.b2.b3.b4 dev eth4 scope link src s41.s42.s43.s44
> > broadcast n1.n2.n3.n4 dev eth4 scope link src s41.s42.s43.s44
> > n1.n2.n3.n4/m dev eth4 scope link src s41.s42.s43.s44
> > default via g1.g2.g3.g4 dev eth4 src s41.s42.s43.s44
>
> You might also want to know that the local routes for eth2-4 are
> removed in the local table, and that the main table holds no default
> routes.
>
> >
> > What am I doing wrong here?
> >
> > TIA
> > /frax

I got help understanding that the source addr is chosen before packet
generated, so the mark is actually matched and the packets routed
through eth4, but with source addr as if they where to be routed
through eth2. So I simply use a another iptables masquerade to get the
right src addr on those specific packets.

/frax

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html