wifi routing dies after 30s

[Patrick Hinchey]

Hi,
I have my brand new VOXL2 with a LTE modem/USB hub, and an Alfa AWUS036ACS wifi dongle plugged into that.
The wifi is configured in station mode.
Both the cellular modem (though it has no SIM card) and the wifi enumerate and give valid IP addresses with ifconfig, and I am able to SSH into the VOXL2 for about 30s after reboot. Then the wifi network routing seems to die. The IP is still listed in ifconfig, and my router web interface still shows the VOXL2 at that address. However I can no longer ping the VOXL2, and on the VOXL2 connected over adb, I can no longer ping the gateway or anything else.
It consistently works right after boot for about 30s.
Any help is much appreciated, thanks!

An excerpt from ifconfig after it has stopped working:

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 2312
        inet 192.168.2.105  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::c74c:4fe1:b608:26a1  prefixlen 64  scopeid 0x20<link>
        ether 00:c0:ca:b1:6c:9f  txqueuelen 1000  (Ethernet)
        RX packets 811  bytes 191916 (191.9 KB)
        RX errors 0  dropped 119  overruns 0  frame 0
        TX packets 99  bytes 15113 (15.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 169.254.174.110  netmask 255.255.0.0  broadcast 169.254.255.255
        inet6 fe80::f4e6:24dc:20b7:ffe0  prefixlen 64  scopeid 0x20<link>
        ether ce:26:e7:9c:b4:6e  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[Steve Turner]

I don't work for ModalAI and I haven't run this specific configuration but a few things to try:

If you remove the LTE Modem does the problem persist? It sounds to me like your default route maybe is changing once the LTE modem comes up.

route -n will show you all of your routes with 0.0.0.0 pointing to the interface associated with it.

[Patrick Hinchey]

Thanks Steve,

Pinging the VOXL2 in one terminal window while continuously executing route -n in adb in another window, I see that first there is nothing in the routing table and the ping says 'host unreachable'. After a few seconds, wlan0 turns up on 0.0.0.0 and 192.168.2.0 in the routing table, and at the same time the ping starts returning a pong.
After about 5 more seconds, wwan0 shows up and everything is still fine. So it looks like the LTE module is not interfering with the WiFi dongle.
Here's where it gets interesting. After another few seconds, usb0 appears on 192.168.2.0 and then the ping return 'host unreachable'. What's going on here?!
It should be noted that the network connectivity craps out even when the VOXL2
is not connected via USB, so the timing might be a coincidence. But it is repeatable.
This behaviour is also common across two different VOXL2 boards

[Patrick Hinchey]

Aha, got it!
So usb0 is interfering with WiFi because my route gateway is 192.168.2.1 and the interface usb0 insists on using 192.168.2.0. If I create use softap mode on the VOXL2 or create a hotspot with my laptop with a different gateway and connect to that, everything works fine.
So now connecting back to my home router and using adb to execute ip link set usb0 down, everything works on the main network!

Now the question is: this solution doesn't persist across reboot, so how can I force usb0 down on boot? Creating/etc/network/interfaces and setting usb to a different address doesn't seem to have any effect. I tried to run ip link set usb0 down in ~/.bashrc but this happens before usb0 goes up and it complains that no such interface exists.

I saw something about voxl-modem, but I could only find info on microhard and feather modems. I couldn't find any documentation on using voxl-modem to disable usb0
@tom how can?

[Patrick Hinchey]

Hopefully someone will reply on Monday with a cleaner approach, but I had some time on Saturday, so I've seconded voxl-static-ip in a hacky way. This has enabled me to keep working through the weekend.

First I just tried to set the IP of usb0 to a 192.168.3.3, but that created a second interface with name usb0, it didn't replace the old one! So in the end I just commented out ModalAI's lines about setting the IP and added ip link set usb0 down instead.

I needed to make the service restart on fail, as it runs before usb0 appears, which causes it to fail.
step by step:

1. edit /usr/bin/voxl-static-ip
   Comment out the last two lines and add one to look like this:

# set new IP
#sudo ifconfig $INTERFACE $LOCAL_IP netmask $NETMASK
#sudo route add default gw $ROUTER_IP $INTERFACE
ip link set usb0 down

2. edit /etc/systemd/system/voxl-static-ip.service
   Add two lines to make the [Service] section look like this:

[Service]
ExecStart=/usr/bin/voxl-static-ip
Restart=on-failure
RestartSec=1s

3. -Profit

[tom]

@Patrick-Hinchey I'm not entirely sure where usb0 is coming from and why it's enumerating. From what I can tell, all you have connected hardware-wise is the WiFi adapter (wlan0) and the LTE modem (wwan0), is that correct?

[Patrick Hinchey]

@tom yep, the LTE modem (wwan0) is plugged into J3 and the Alfa WiFi module (wlan0) is plugged into the LTE modem's USB breakout. I don't understand why usb0 would be there either - at first I thought it the adb but adb continues to work even when I set usb0 down.

FWIW, I bought multiple units, and so far both the ones I've tried enumerate usb0 on both platform releases 1.1 and 1.3

[tom]

@Patrick-Hinchey From what I can see, it looks like it's a bug in voxl-modem where usb0 is being pulled up when it shouldn't be. I'll have to dig into this a bit to find root cause.

It is somewhere in /usr/bin/voxl-modem-start.sh that is doing it. This is the modem enumeration script that runs at boot. You can see the systemd service file at /etc/systemd/system/voxl-modem.service.

[tom]

This post is deleted!

[Patrick Hinchey]

Interesting! I'll keep watching this thread for when you commit a proper fix in voxl-modem. For the meantime, my hack is working for me. Thanks!