In order to access the webserver for the radio you intend to install onboard the voxl2 you should first access the webserver when it's installed on the host pc carrier board and change the firewall settings to enable remote access. The ip address that's set as the static ip in LAN settings is the actual radio ip address. You enter that address in your browser to access the webserver, not the ip address you assigned when running voxl-configure-modem.

I recommend creating a free account on the microhard support website and downloading the product manual here:

https://support.microhardcorp.com/portal/en/kb/articles/pmddl2450-pmddl2350-pmddl2550-user-manual

It's got a pretty good walkthrough on setting up a master/slave radio configuration right up front.

Here's a pic of the general firewall settings as I have them configured. When I first tried to set up the radios I did so on the host pc carrier but when I deployed them to the voxl2 I couldn't access the webserver either. I realized the remote access is disabled by default, you have to check that middle box.

Hope this helps.file:///home/joe/Pictures/Screenshot%20from%202023-07-31%2009-37-33.png

Found the source of the build issues. This line in voxl-mpa-to-ros2

python3 -m pip install colcon-common-extensions lark

is installing an incompatible version of empy. See here:

https://github.com/colcon/colcon-core/issues/602

Another issue with a default sentinel/voxl2 set-up when installing ros2 I've found - need to add a sim link for libtinyxml2. I got this build error

make[2]: *** No rule to make target '/usr/lib/aarch64-linux-gnu/libtinyxml2.so', needed by 'libpx4_msgs__rosidl_typesupport_fastrtps_cpp.so'.  Stop

which I resolved via

ln -s /usr/lib/aarch64-linux-gnu/libtinyxml2.so.6 /usr/lib/aarch64-linux-gnu/libtinyxml2.so

I don't know if these issues are specific to the sentinel platform or not. I added the empy version fix to a fork of voxl-mpa-to-ros2 and will push a merge request. I'm guessing there should be a similar fix within a voxl-ros2-foxy install script or else there'll be build issues whenever trying to build a ros package onboard a vehicle that compiles ros messages (empy version issue was what was preventing me from building px4_msgs package onboard)

@Aaron-Porter I'm sure the system id fix might have resolved the issue. I just find having separate ports for each vehicle to establish a connection over a cleaner option. This isn't an issue for me anymore, I was just responding to the question on the approach I used to resolve it

I just disconnected/reconnected the USB to 4-pin JST cable from the microhard carrier board and rebooted the vehicle. Now the wlan0 interface has been up the entire time the vehicle's on, but none of the other devices on the vLAN the wifi adapter is connected to can ping the vehicle over wifi.

Output from dmesg looks the same except the following output after the microhard interface is found:

[   26.460342] usb 1-1.2: new high-speed USB device number 4 using xhci-hcd
[   26.550901] usb 1-1.2: New USB device found, idVendor=1f94, idProduct=3002, bcdDevice= 4.14
[   26.550915] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   26.550922] usb 1-1.2: Product: pMDDL2450AES256
[   26.550929] usb 1-1.2: Manufacturer: Microhard
[   26.550935] usb 1-1.2: SerialNumber: 1592348
[   26.639101] rndis_host 1-1.2:1.0 usb0: register 'rndis_host' at usb-xhci-hcd.0.auto-1.2, RNDIS device, 16:4f:08:e4:e1:09
[   26.643401] cdc_acm 1-1.2:1.2: ttyACM0: USB ACM device
[   33.765387] vdd_tof: disabling
[   33.765393] vdd_hap_boost: disabling
[   37.240138] hrtimer: interrupt took 112083 ns
[  101.412343] boot log copy done
[  114.937520] qcom_rpmh DRV:apps_rsc TCS Busy, retrying RPMH message send: addr=0x41b08
[  233.747906] perf: interrupt took too long (2778 > 2500), lowering kernel.perf_event_max_sample_rate to 72000
[  339.990578] perf: interrupt took too long (3473 > 3472), lowering kernel.perf_event_max_sample_rate to 57500
[  386.060248] perf: interrupt took too long (4425 > 4341), lowering kernel.perf_event_max_sample_rate to 45000

I have multiple Sentinels with microhard add-on boards and wifi adapters connected to the USB peripheral port on the microhard carrier board (not custom, this is how they were shipped). Wifi is working without issue on 3/4 of the vehicles, but for one of the vehicles the wifi adapter is either never recognized after booting up or it is detected and works for a few minutes but then disconnects and will not reestablish a connection.

I used the instructions here to configure dhcpcd to set a static ip adress for the wlan0 interface for all the vehicles:
https://forum.modalai.com/topic/4105/instruction-to-set-a-static-ip-for-wlan0-voxl2

It works without issue for the other vehicles so I don't think it's part of the problem. I've attached the output of dmesg following a boot experiencing these issues and filtered the output for lines that contain "usb". You can see the wifi adapter is detected around 7 seconds after boot, the microhard interface is detected around 26 seconds after boot, but around 281 s after boot the wifi adapter starts diconnecting and can't be enabled.

Can me someone help me troubleshoot this? I'm assuming at this point since the configurations are the same on all the other vehicles that this is most likely a hardware issue. How can I confirm whether it is or not?

[    0.000000] Kernel command line: rcupdate.rcu_expedited=1 rcu_nocbs=0-7 kpti=off noinitrd  earlycon=msm_geni_serial,0xa90000 androidboot.hardware=qcom androidboot.console=ttyMSM0 androidboot.memcg=1 lpm_levels.sleep_disabled=1 video=vfb:640x400,bpp=32,memsize=3072000 msm_rtb.filter=0x237 service_locator.enable=1 androidboot.usbcontroller=a600000.dwc3 swiotlb=2048 loop.max_part=7 cgroup.memory=nokmem,nosocket reboot=panic_warm net.ifnames=0 usbcore.autosuspend=-1 apparmor=1 security=apparmor root=/dev/sda9 androidboot.bootdevice=1d84000.ufshc androidboot.serialno=75caf339 androidboot.baseband=apq msm_drm.dsi_display0=qcom,mdss_dsi_ext_bridge_1080p: systemd.setenv="SLOT_SUFFIX=_a" skip_initramfs rootwait rw init=/sbin/init
[    1.672769] usbcore: registered new interface driver usbfs
[    1.672790] usbcore: registered new interface driver hub
[    1.672883] usbcore: registered new device driver usb
[    1.673141] usb_phy_generic soc:usb_nop_phy: soc:usb_nop_phy supply vcc not found, using dummy regulator
[    1.673170] usb_phy_generic soc:usb_nop_phy: Linked as a consumer to regulator.0
[    1.857352] usbcore: registered new interface driver r8152
[    1.857366] usbcore: registered new interface driver lan78xx
[    1.857374] usbcore: registered new interface driver asix
[    1.857380] usbcore: registered new interface driver ax88179_178a
[    1.857385] usbcore: registered new interface driver cdc_ether
[    1.857391] usbcore: registered new interface driver cdc_eem
[    1.857399] usbcore: registered new interface driver smsc75xx
[    1.857406] usbcore: registered new interface driver smsc95xx
[    1.857412] usbcore: registered new interface driver net1080
[    1.857418] usbcore: registered new interface driver rndis_host
[    1.857422] usbcore: registered new interface driver cdc_subset
[    1.857428] usbcore: registered new interface driver zaurus
[    1.857434] usbcore: registered new interface driver sierra_net
[    1.857440] usbcore: registered new interface driver cdc_ncm
[    1.857446] usbcore: registered new interface driver qmi_wwan
[    1.857451] usbcore: registered new interface driver cdc_mbim
[    1.857698] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.32
[    1.857712] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.18
[    1.857718] msm-usb-ssphy-qmp 88e8000.ssphy: failed to get phandle for msm_ssphy_qmp
[    1.857744] msm-usb-ssphy-qmp 88e8000.ssphy: Dropping the link to regulator.18
[    1.857759] msm-usb-ssphy-qmp 88e8000.ssphy: Dropping the link to regulator.32
[    1.857837] msm-usb-ssphy-qmp 88eb000.ssphy: Linked as a consumer to regulator.32
[    1.857847] msm-usb-ssphy-qmp 88eb000.ssphy: Linked as a consumer to regulator.18
[    1.857999] msm-usb-hsphy 88e3000.hsphy: Linked as a consumer to regulator.13
[    1.858021] msm-usb-hsphy 88e3000.hsphy: Linked as a consumer to regulator.10
[    1.858034] msm-usb-hsphy 88e3000.hsphy: Linked as a consumer to regulator.21
[    1.858211] msm-usb-hsphy 88e4000.hsphy: Linked as a consumer to regulator.13
[    1.858222] msm-usb-hsphy 88e4000.hsphy: Linked as a consumer to regulator.10
[    1.858232] msm-usb-hsphy 88e4000.hsphy: Linked as a consumer to regulator.21
[    1.858778] msm-dwc3 a600000.ssusb: Linked as a consumer to 15000000.apps-smmu
[    1.859004] iommu: Adding device a600000.ssusb to group 13
[    1.859152] msm-dwc3 a600000.ssusb: Linked as a consumer to regulator.66
[    1.860588] msm-dwc3 a600000.ssusb: unable to get ssphy device
[    1.861600] msm-dwc3 a600000.ssusb: Dropping the link to regulator.66
[    1.861655] msm-dwc3 a800000.ssusb: Linked as a consumer to 15000000.apps-smmu
[    1.861848] iommu: Adding device a800000.ssusb to group 14
[    1.862001] msm-dwc3 a800000.ssusb: Linked as a consumer to regulator.67
[    1.863002] register_client_adhoc:Client handle 36 usb1
[    1.863723] usbcore: registered new interface driver cdc_acm
[    1.863732] usbcore: registered new interface driver cdc_wdm
[    1.863741] usbcore: registered new interface driver usb-storage
[    1.863756] usbcore: registered new interface driver cp210x
[    1.863761] usbserial: USB Serial support registered for cp210x
[    1.863767] usbcore: registered new interface driver ftdi_sio
[    1.863771] usbserial: USB Serial support registered for FTDI USB Serial Device
[    1.863778] usbcore: registered new interface driver option
[    1.863782] usbserial: USB Serial support registered for GSM modem (1-port)
[    1.863787] usbcore: registered new interface driver qcserial
[    1.863791] usbserial: USB Serial support registered for Qualcomm USB modem
[    1.863797] usbcore: registered new interface driver sierra
[    1.863801] usbserial: USB Serial support registered for Sierra USB modem
[    1.863807] usbcore: registered new interface driver usb_ehset_test
[    1.863813] usbcore: registered new interface driver lvs
[    1.864094] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: Linked as a consumer to regulator.10
[    1.864626] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: usbpd_create failed: -517
[    1.864693] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: Dropping the link to regulator.10
[    1.864840] usbcore: registered new interface driver xpad
[    1.864847] usbcore: registered new interface driver usb_acecad
[    1.864852] usbcore: registered new interface driver aiptek
[    1.864858] usbcore: registered new interface driver gtco
[    1.864864] usbcore: registered new interface driver hanwang
[    1.864868] usbcore: registered new interface driver kbtab
[    1.871415] ssusb redriver 3-001c: Could not get usb power_supply
[    1.871420] ssusb redriver 3-001c: failed to get phandle for redriver
[    1.877138] usbcore: registered new interface driver uvcvideo
[    1.893434] usbcore: registered new interface driver usbhid
[    1.893436] usbhid: USB HID core driver
[    1.907420] usbcore: registered new interface driver snd-usb-audio
[    1.907510] uaudio-qmi soc:usb_audio_qmi_dev: Linked as a consumer to 15000000.apps-smmu
[    1.907534] iommu: Adding device soc:usb_audio_qmi_dev to group 20
[    1.920303] msm-dwc3 a800000.ssusb: DWC3 exited from low power mode
[    1.921251] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 4.19
[    1.921264] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.921274] usb usb1: Product: xHCI Host Controller
[    1.921281] usb usb1: Manufacturer: Linux 4.19.125 xhci-hcd
[    1.921288] usb usb1: SerialNumber: xhci-hcd.0.auto
[    1.922122] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 4.19
[    1.922134] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.922145] usb usb2: Product: xHCI Host Controller
[    1.922152] usb usb2: Manufacturer: Linux 4.19.125 xhci-hcd
[    1.922160] usb usb2: SerialNumber: xhci-hcd.0.auto
[    1.959048] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.32
[    1.959075] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.18
[    1.959080] msm-usb-ssphy-qmp 88e8000.ssphy: failed to get phandle for msm_ssphy_qmp
[    1.959099] msm-usb-ssphy-qmp 88e8000.ssphy: Dropping the link to regulator.18
[    1.959109] msm-usb-ssphy-qmp 88e8000.ssphy: Dropping the link to regulator.32
[    1.959186] msm-dwc3 a600000.ssusb: Linked as a consumer to regulator.66
[    1.960319] usb_bam_init: setting SPS_BAM_SMMU_EN flag with (a704000.qcom,usbbam)
[    1.960427] msm-dwc3 a600000.ssusb: unable to get ssphy device
[    1.961090] msm-dwc3 a600000.ssusb: Dropping the link to regulator.66
[    1.961154] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: Linked as a consumer to regulator.10
[    1.961704] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: usbpd_create failed: -517
[    1.961777] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: Dropping the link to regulator.10
[    1.967371] ssusb redriver 3-001c: Could not get usb power_supply
[    1.967375] ssusb redriver 3-001c: failed to get phandle for redriver
[    1.977320] SMB5 status - usb:present=0 type=0 batt:present = 1 health = 1 charge = 1
[    1.989400] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.32
[    1.989413] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.18
[    1.989418] msm-usb-ssphy-qmp 88e8000.ssphy: failed to get phandle for msm_ssphy_qmp
[    1.989433] msm-usb-ssphy-qmp 88e8000.ssphy: Dropping the link to regulator.18
[    1.989441] msm-usb-ssphy-qmp 88e8000.ssphy: Dropping the link to regulator.32
[    1.989517] msm-dwc3 a600000.ssusb: Linked as a consumer to regulator.66
[    1.990404] usb_bam_init: setting SPS_BAM_SMMU_EN flag with (a704000.qcom,usbbam)
[    1.990508] msm-dwc3 a600000.ssusb: unable to get ssphy device
[    1.991138] msm-dwc3 a600000.ssusb: Dropping the link to regulator.66
[    1.991202] qpnp-pdphy c440000.qcom,spmi:qcom,pm8150b@2:qcom,usb-pdphy@1700: Linked as a consumer to regulator.10
[    1.997331] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.32
[    1.997345] msm-usb-ssphy-qmp 88e8000.ssphy: Linked as a consumer to regulator.18
[    1.997349] msm-usb-ssphy-qmp 88e8000.ssphy: No separate ID extcon device
[    1.997392] msm-dwc3 a600000.ssusb: Linked as a consumer to regulator.66
[    1.998227] usb_bam_init: setting SPS_BAM_SMMU_EN flag with (a704000.qcom,usbbam)
[    1.998632] register_client_adhoc:Client handle 63 usb0
[    1.998672] msm-dwc3 a600000.ssusb: Linked as a consumer to regulator.86
[    2.056179] msm-dwc3 a600000.ssusb: DWC3 exited from low power mode
[    2.056930] msm-dwc3 a600000.ssusb: DWC3 in low power mode
[    2.287804] usbcore: registered new interface driver 8188eu
[    2.327455] usbcore: registered new interface driver rtl8821cu
[    2.346036] usbcore: registered new interface driver rtl88XXau
[    2.656045] usb 1-1: new high-speed USB device number 2 using xhci-hcd
[    2.792395] usb 1-1: New USB device found, idVendor=0424, idProduct=2514, bcdDevice= b.b3
[    2.792401] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    3.184140] usb 1-1.4: new high-speed USB device number 3 using xhci-hcd
[    4.016323] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[    4.852415] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[    4.852644] usb 1-1-port4: attempt power cycle
[    5.018757] f_cdev_alloc: port_name:at_usb0 (ffffffe38c15a800) portno:(0)
[    5.021717] f_cdev_alloc: port_name:at_usb1 (ffffffe38c158400) portno:(1)
[    6.016434] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[    6.088053] usb 1-1.4: new high-speed USB device number 6 using xhci-hcd
[    6.088136] usb 1-1.4: Device not responding to setup address.
[    6.297149] usb 1-1.4: Device not responding to setup address.
[    6.500946] usb 1-1.4: device not accepting address 6, error -71
[    6.501376] usb 1-1-port4: unable to enumerate USB device
[    6.921975] usb 1-1.4: new high-speed USB device number 7 using xhci-hcd
[    7.029686] usb 1-1.4: New USB device found, idVendor=0bda, idProduct=0811, bcdDevice= 2.00
[    7.029695] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    7.029700] usb 1-1.4: Product: 802.11ac WLAN Adapter 
[    7.029705] usb 1-1.4: Manufacturer: Realtek 
[    7.029709] usb 1-1.4: SerialNumber: 00e04c000001
[    7.274284] usb 1-1.4: 88XXau 00:c0:ca:b5:ab:9c hw_info[107]
[   26.444557] usb 1-1.2: new high-speed USB device number 8 using xhci-hcd
[   26.534908] usb 1-1.2: New USB device found, idVendor=1f94, idProduct=3002, bcdDevice= 4.14
[   26.534925] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   26.534934] usb 1-1.2: Product: pMDDL2450AES256
[   26.534942] usb 1-1.2: Manufacturer: Microhard
[   26.534951] usb 1-1.2: SerialNumber: 1592348
[   26.634655] rndis_host 1-1.2:1.0 usb0: register 'rndis_host' at usb-xhci-hcd.0.auto-1.2, RNDIS device, 06:f2:15:a9:a5:b2
[  281.219466] usb 1-1.4: USB disconnect, device number 7
[  281.515539] usb 1-1.4: new high-speed USB device number 9 using xhci-hcd
[  281.605319] usb 1-1.4: New USB device found, idVendor=0bda, idProduct=0811, bcdDevice= 2.00
[  281.605329] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  281.605333] usb 1-1.4: Product: 802.11ac WLAN Adapter 
[  281.605337] usb 1-1.4: Manufacturer: Realtek 
[  281.605341] usb 1-1.4: SerialNumber: 00e04c000001
[  281.932340] usb 1-1.4: 88XXau 00:c0:ca:b5:ab:9c hw_info[107]
[  297.859348] usb 1-1.4: USB disconnect, device number 9
[  298.159693] usb 1-1.4: new high-speed USB device number 10 using xhci-hcd
[  298.249415] usb 1-1.4: New USB device found, idVendor=0bda, idProduct=0811, bcdDevice= 2.00
[  298.249426] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  298.249432] usb 1-1.4: Product: 802.11ac WLAN Adapter 
[  298.249437] usb 1-1.4: Manufacturer: Realtek 
[  298.249442] usb 1-1.4: SerialNumber: 00e04c000001
[  298.574671] usb 1-1.4: 88XXau 00:c0:ca:b5:ab:9c hw_info[107]
[  300.419153] usb 1-1.4: USB disconnect, device number 10
[  300.684351] usb 1-1.4: new high-speed USB device number 11 using xhci-hcd
[  301.534222] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[  301.607787] usb 1-1.4: new high-speed USB device number 12 using xhci-hcd
[  302.440236] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[  302.440408] usb 1-1-port4: attempt power cycle
[  303.580855] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[  304.428990] usb 1-1-port4: Cannot enable. Maybe the USB cable is bad?
[  304.430266] usb 1-1-port4: unable to enumerate USB device

@ashwin I just added a parameter called qgc_udp_port to config_file.h/c in voxl-mavlink-server (you can use the same syntax as used for any of the other port number variables in the config file to add it) and then added some logic to the gcs_io.c file where it makes the connections and rebuilt voxl-mavlink-server locally. I don't have that version somewhere publically available, but the required changes are so minor you could easily update and rebuild voxl-mavlink-server yourself

In the gcs_io_init(void) function in gcs_io.c you could replace the line

si_me_gcs.sin_port = htons(GCS_UDP_PORT);

with something like

si_me_gcs.sin_port = htons((uint16_t)qgc_udp_port);

where qgc_udp_port is the variable you add to the config file. You'd also need to go the _add_connection(unsigned long ip) function and replace

connections[i].sockaddr.sin_port = htons(GCS_UDP_PORT);

with something like

connections[i].sockaddr.sin_port = htons((uint16_t)gcs_udp_publish_port);

I also added some logic to check that the user generated port number is within a valid range and only change it for each connection if it's not equal to the value defined by GCS_UDP_PORT (14550), but that's not strictly necessary.

This was all done for voxl-mavlink-server in SDK 1.3.5. I don't know how much the logic has changed in the more recent SDK versions, but I imagine these changes would carry over

Is there a way to expose real-time signal strength data from the microhard radios? Either to use in post for data analysis or for use in real-time as an input for some algorithm

@Eric-Katzfey Awesome, thanks for the feedback!

Will these instructions work for a sentinel dev drone?

https://docs.modalai.com/voxl2-PX4-hitl/

Is the J18 port exposed? Specifically for sentinels with the mircohard carrier board add on

Also, does modalai sell the FTDI cable I would need to run HITL?

Thanks

I'm finding it impossible to run ros2 nodes on multiple vehicles networked together without using the fastdds discovery server to limit traffic to unicast and cut down discovery traffic. Using the default ros2 / fast dds set-up floods the network with traffic once add a third vehicle to the network. Only problem with the discovery server is it isolates the vehicle nodes from the px4 topics exposed by the microdds agent, which I need to publish to.

Is there a way to bridge the microdds agent and nodes that are connected to a dds discovery server? The command line options for running the agent seem pretty limited (can specify discover port but not address) and it's not clear to me how the default agent discovery even works (unicast/multicast/both) and whether or not it's configurable

Has anyone encountered latency issues / stalls in data streams when running ROS2 nodes on multiple vehicles? I'm running ros nodes to publish mocap data as odometry data to sentinel vehicles and am seeing significant performance degradation whenever I have more than 2 vehicles running (see here for more details https://forum.modalai.com/topic/4018/issues-with-multiple-vehicles-on-microhard-network)

I'm reading through the dds documentation to figure out how to tune it to reduce network traffic. Using a discovery server seems like a good option but then it cuts off communication with the microdds-agent and I'm not sure how to get around that. I'm modifying a dds xml profile to try to reduce rtps traffic, but I'm still having issues.

If anyone else has encountered similar issues I'd appreciate any recommendations, thanks!

I have multiple Sentinels (SDK 1.3.5) with Mircohard pmddl2450 radios and wifi dongles that I'm attempting to perform indoor flight testing with. I'm using ROS2 nodes to stream mocap data and joystick commands to vehicles nodes via the microhard network. I've had the network set up as point to multipoint with the groundstation radio in master mode and I've had the network set up as a mesh.

With both set-ups everything performs well for up to two vehicles, but as soon as I add a third vehicle performs starts to degrade significantly and is unusable with a fourth vehicle. Mocap data being streamed to the vehicles at 120 Hz is being received between 15-60 Hz by the vehicles with significant stalls (1-2s) throughout operation. The total amount of data being transmitted to the vehicles is about 2Mbps for mocap data alone (500 kbps per vehicle). The px4 data being streamed over the network to the gcs for qgc is only about 25 kbps. I moved that over to wifi to try to clear up some bandwidth but that didn't help.

I've read that this can be a common issue with default settings for the DDS implementation for ros2 so I've been tuning the dds configs to no avail. I decided to write a simple test program sending the same amount of data to the vehicles over the microhard network via zeromq TCP sockets and I'm still seeing significantly degraded performance (latency > 30ms and intermittent stalls) for more than two vehicles.

Should I expect this network topology to work (single ground station radio connected to 4+ vehicle radios) ? The pmddl2450 is marketed as supporting 25 Mbps and I'm not coming close to that. I'm not sure how to continue troubleshooting this so any help/advice is appreciated