Re: [LEDE-DEV] DHCP via bridge in case of IPv4

[Alexey Brodkin]

Hello,

On Mon, 2016-07-11 at 06:15 +0000, Alexey Brodkin wrote:
> Hi Russel,
> 
> On Sun, 2016-07-10 at 00:19 -0700, Russell Senior wrote:
> >Â
> > > > > > > "Alexey" == Alexey Brodkin <Alexey.Brodkin@xxxxxxxxxxxx> writes:
> > Alexey> Hi Aaron,
> > Alexey> On Sat, 2016-07-09 at 07:47 -0400, Aaron Z wrote:
> > >Â
> > > > On Sat, Jul 9, 2016 at 4:37 AM, Alexey Brodkin
> > > > <Alexey.Brodkin@xxxxxxxxxxxx> wrote:
> > > > >Â
> > > > > Hello,
> > > > > 
> > > > > I was playing with quite simple bridged setup on different boards
> > > > with > very recent kernels (4.6.3 as of this writing) and found one
> > > > interesting > behavior that I cannot yet understand and googling
> > > > din't help here as well.
> > > > >Â
> > > > > My setup is pretty simple: >
> > > > -------------ÂÂÂÂÂÂÂ------------------ÂÂÂÂÂÂÂ-------------------------
> > > > >Â
> > > > > > HOSTÂÂÂÂÂÂ|ÂÂÂÂÂÂÂ| "Dumb AP"ÂÂÂÂÂÂ|ÂÂÂÂÂÂÂ| Wireless
> > > > clientÂÂÂÂÂÂÂ| > > with DHCP |<----->(eth0)ÂÂÂÂÂ(wlan0)<----->|
> > > > attempting toÂÂÂÂÂÂÂÂÂ| > > serverÂÂÂÂ|ÂÂÂÂÂÂÂ|ÂÂÂÂ\ br0
> > > > /ÂÂÂÂÂ|ÂÂÂÂÂÂÂ| get settings via DHCP | >
> > > > -------------ÂÂÂÂÂÂÂ------------------ÂÂÂÂÂÂÂ-------------------------
> > > > > * HOST is my laptop with DHCP server that works for sure.ÂÂ> *
> > > > "Dumb AP" is a separate board (I tried ARM-based Wandboard and
> > > > ARC-based > Â AXS10x boards but results are exactly the same) with
> > > > wired (eth0) and wireless > Â (wlan0) network controllers bridged
> > > > together (br0). That "br0" bridge flawlessly > Â gets its settings
> > > > from DHCP server on host.ÂÂ> * Wireless client could be either a
> > > > smatrphone or another laptop etc but > Â what's important it should
> > > > be configured to get network settings by DHCP as well.
> > > > > 
> > > > > So what happens "br0" always gets network settings from DHCP server
> > > > on HOST.ÂÂ> That's fine. But wireless client only reliably gets
> > > > settings from DHCP server > if IPv6 is enabled on "Dumb AP" board. If
> > > > IPv6 is disabled I may see that > wireless client sends "DHCP
> > > > Discover" then server replies with "DHCP Offer" but > that offer
> > > > never reaches wireless client.
> > > > 
> > > > Do you have WDS enabled? If not, DHCP has issues in that scenario:
> > > > https://wiki.openwrt.org/doc/howto/clientmode
> > If the Dumb AP's wireless interface is in ap-mode, then this shouldn't
> > be an issue.ÂÂIt's only client-mode interfaces that have trouble with bridging.
> > 
> > I'd suggest running tcpdump on the Dumb AP's wireless interface and the
> > client's wireless interface and see which of them sees the various parts
> > of the DHCP handshake.
> 
> So I did but for DHCP server and wireless client (had no tcpdump on Dump AP
> at the moment).
> 
> That's what I see on the server:
> ----------------------------->8-------------------------------
> No. TimeÂÂÂÂÂÂÂÂSourceÂÂÂÂÂDestinationÂÂÂÂÂÂProtocol Length Info
> Â3 0.151181000ÂÂ0.0.0.0ÂÂÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 11 2.760796000ÂÂ10.42.0.1ÂÂ10.42.0.13ÂÂÂÂÂÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP OfferÂÂÂÂ- Transaction ID 0x31dc321f
> 14 5.220985000ÂÂ0.0.0.0ÂÂÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 15 5.221150000ÂÂ10.42.0.1ÂÂ10.42.0.13ÂÂÂÂÂÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP OfferÂÂÂÂ- Transaction ID 0x31dc321f
> 23 15.649835000 0.0.0.0ÂÂÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 24 15.650017000 10.42.0.1ÂÂ10.42.0.13ÂÂÂÂÂÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP OfferÂÂÂÂ- Transaction ID 0x31dc321f
> 32 25.648589000 0.0.0.0ÂÂÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 33 25.648758000 10.42.0.1ÂÂ10.42.0.13ÂÂÂÂÂÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP OfferÂÂÂÂ- Transaction ID 0x31dc321f
> 43 35.864567000 0.0.0.0ÂÂÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 48 38.832837000 10.42.0.1ÂÂ10.42.0.13ÂÂÂÂÂÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP OfferÂÂÂÂ- Transaction ID 0x31dc321f
> ----------------------------->8-------------------------------
> 
> That's on the wireless client:
> ----------------------------->8-------------------------------
> No.ÂÂTimeÂÂÂÂÂÂÂÂÂÂÂSourceÂÂÂDestinationÂÂÂÂÂÂProtocol Length Info
> 1171 94.192971000ÂÂÂ0.0.0.0ÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 1182 99.263686000ÂÂÂ0.0.0.0ÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 1185 109.692642000ÂÂ0.0.0.0ÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 1186 119.691474000ÂÂ0.0.0.0ÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> 1190 129.907507000ÂÂ0.0.0.0ÂÂ255.255.255.255ÂÂDHCPÂÂÂÂÂ342ÂÂÂÂDHCP Discover - Transaction ID 0x31dc321f
> ----------------------------->8-------------------------------
> 
> I'll try to capture data from Dumb AP sometime soon and will reply to the thread.

So finally after quite some time I figured out what happens in my setup.
Basically it all boils down to the fact that DW GMAC doesn't enter promiscuous mode
when bridge gets created. To be more precise GMAC's MAC filter fail to enter promiscuous mode
and so packets from DHCP server sent to Wi-Fi clien are discarded by GMAC - that's why I cannot
see those packets in tcpdump output - CPU never gets any information about them.

I noticed that problem only happens if Ethernet PHY (in my case this is DP83865) doesn't have
link established. I.e. either:
Â1) Cable is disconnected
Â2) PHY is still negotiation link mode

Once PHY got link established GMAC happily enters promisq mode and all works as expected.

Unfortunately I cannot figure out why GMAC behaves that way. As per GMAC's databook the only reason
for it to not react on a register being written is missing input clock but there's no reason for the
clock to be missing and I don't seem to have a way to check if there's a problem with clock or not.

-Alexey