Using Relocalization to establish a global reference frame for Drone Swarm Demo
-
Hello ModalAI Team,
we currently are trying to prepare an indoor demo with multiple Qualcomm Flight RB5 drones. First i will share the configuration currently on the drones:system-image: 1.7.1-M0052-14.1a-perf-nightly-20231025 kernel: #1 SMP PREEMPT Thu Oct 26 05:24:02 UTC 2023 4.19.125 -------------------------------------------------------------------------------- hw version: M0052 -------------------------------------------------------------------------------- voxl-suite: 1.1.2 -------------------------------------------------------------------------------- current IP: wlan0: 192.168.1.34 --------------------------------------------------------------------------------Output of voxl-inspect-services
Service Name | Enabled | Running | CPU Usage ------------------------------------------------------------------- docker-autorun | Disabled | Not Running | modallink-relink | Disabled | Not Running | voxl-camera-server | Enabled | Running | 63.1% voxl-cpu-monitor | Enabled | Running | 0.5% voxl-dfs-server | Disabled | Not Running | voxl-feature-tracker | Enabled | Not Running | voxl-flow-server | Disabled | Not Running | voxl-imu-server | Disabled | Not Running | voxl-lepton-server | Disabled | Not Running | voxl-mavcam-manager | Enabled | Running | 0.0% voxl-mavlink-server | Enabled | Running | 2.7% voxl-microdds-agent | Enabled | Running | 5.6% voxl-modem | Disabled | Not Running | voxl-neopixel-manager | Disabled | Not Running | voxl-portal | Enabled | Running | 0.1% voxl-px4-imu-server | Enabled | Running | 0.9% voxl-px4 | Enabled | Running | 38.4% voxl-qvio-server | Enabled | Running | 20.4% voxl-rangefinder-server | Disabled | Not Running | voxl-remote-id | Disabled | Not Running | voxl-softap | Disabled | Not Running | voxl-static-ip | Disabled | Not Running | voxl-streamer | Enabled | Running | 0.1% voxl-tag-detector | Enabled | Running | 4.2% voxl-tflite-server | Disabled | Not Running | voxl-time-sync | Disabled | Not Running | voxl-uvc-server | Disabled | Not Running | voxl-vision-hub | Enabled | Running | 1.8% voxl-wait-for-fs | Enabled | Completed |Since the drones are going to fly indoors, they will operate in a GPS-denied area. We are already able to fly the drone in position mode with VIO tracking. For the use case of the demo however, this will not be enough, because we will need the drones to be positioned in a global coordinate system.
The first approach to resolve this issue was trying out the Relocalization with April Tags. The voxl-vision-hub.conf file looks like this at the moment:{ "config_file_version": 1, "en_localhost_mavlink_udp": true, "localhost_udp_port_number": 14551, "en_vio": true, "vio_pipe": "qvio", "secondary_vio_pipe": "ov", "en_reset_vio_if_initialized_inverted": true, "vio_warmup_s": 3, "send_odom_while_failed": true, "horizon_cal_tolerance": 0.5, "offboard_mode": "off", "follow_tag_id": 0, "figure_eight_move_home": true, "robot_radius": 0.300000011920929, "collision_sampling_dt": 0.1, "max_lookahead_distance": 1, "en_tag_fixed_frame": true, "fixed_frame_filter_len": 5, "en_transform_mavlink_pos_setpoints_from_fixed_frame": true, "en_voa": true, "voa_upper_bound_m": -0.15000000596046448, "voa_lower_bound_m": 0.15000000596046448, "voa_voa_memory_s": 1, "voa_max_pc_per_fusion": 100, "voa_pie_max_dist_m": 20, "voa_pie_min_dist_m": 0.25, "voa_pie_under_trim_m": 1, "voa_pie_threshold": 3, "voa_send_rate_hz": 20, "voa_pie_slices": 36, "voa_pie_bin_depth_m": 0.15000000596046448, "voa_inputs": [{ "enabled": true, "type": "point_cloud", "input_pipe": "dfs_point_cloud", "frame": "stereo_l", "max_depth": 8, "min_depth": 0.300000011920929, "cell_size": 0.079999998211860657, "threshold": 4, "x_fov_deg": 68, "y_fov_deg": 56, "conf_cutoff": 0 }, { "enabled": true, "type": "point_cloud", "input_pipe": "stereo_front_pc", "frame": "stereo_front_l", "max_depth": 8, "min_depth": 0.300000011920929, "cell_size": 0.079999998211860657, "threshold": 4, "x_fov_deg": 68, "y_fov_deg": 56, "conf_cutoff": 0 }, { "enabled": true, "type": "point_cloud", "input_pipe": "stereo_rear_pc", "frame": "stereo_rear_l", "max_depth": 8, "min_depth": 0.300000011920929, "cell_size": 0.079999998211860657, "threshold": 4, "x_fov_deg": 68, "y_fov_deg": 56, "conf_cutoff": 0 }, { "enabled": true, "type": "tof", "input_pipe": "tof", "frame": "tof", "max_depth": 6, "min_depth": 0.15000000596046448, "cell_size": 0.079999998211860657, "threshold": 3, "x_fov_deg": 106.5, "y_fov_deg": 85.0999984741211, "conf_cutoff": 125 }, { "enabled": true, "type": "rangefinder", "input_pipe": "rangefinders", "frame": "body", "max_depth": 8, "min_depth": 0.300000011920929, "cell_size": 0.079999998211860657, "threshold": 4, "x_fov_deg": 68, "y_fov_deg": 56, "conf_cutoff": 0 }] }and the tag_locations.conf file looks like this:
{ "locations": [{ "id": 1, "name": "origin", "loc_type": "fixed", "size_m": 0.40000000596046448, "T_tag_wrt_fixed": [0, 0, 0], "R_tag_to_fixed": [[0, -1, 0], [1, 0, 0], [0, 0, 1]] }], "default_size_m": 0.40000000596046448 }For the sake of clarity i removed your comments here. We were able to confirm with the command
voxl-vision-hub -pthat the april tag is correctly recognized and valueT_body_wrt_fixedrelocates to closely [0,0,0] as expected. So we can confirm that the relocalization is internally working.The PC and all Drones are connected with the same WiFi. The goal is to control multiple drones with a single python script using MAVSDK running on the PC. This is where we would need your help
async def run(): drone = System() await drone.connect()The MAVSDK classes and functions are defined as following:
class mavsdk.system.System(mavsdk_server_address=None, port=50051, sysid=245, compid=190) Bases: object Instantiate a System object, that will serve as a proxy to all the MAVSDK plugins. Parameters: mavsdk_server_address (str) – Address of a running mavsdk_server instance. If None, an instance of mavsdk_server will be automatically started (on localhost). port (int) – Port of the running mavsdk_server instance specified by mavsdk_server_address. sysid (int) – MAVLink system ID of the mavsdk_server (1..255). compid (int) – MAVLink component ID of the mavsdk_server (1..255).async connect(system_address=None) Connect the System object to a remote system. Parameters: system_address (str) – The address of the remote system. If None, it will default to udp://:14540. Supported URL formats: Serial: serial:///path/to/serial/dev[:baudrate] UDP: udp://[bind_host][:bind_port] TCP: tcp://[server_host][:server_port]Can you help us which parameters needs to be put in the
System()andconnect()to be able to control a drone in offboard flightmode. Or if we would need to do some adjustments on thevoxl-mavlink-server.
Thank you in advance for your help and let me know if you need any further information on our system/setup.
With kind regards
Julian -
@Quetzalcoatl We have used mavsdk running on VOXL2 (versus on the PC). Some documentation is here: https://docs.modalai.com/mavsdk/. An example Python script is here: https://github.com/mavlink/MAVSDK-Python/blob/main/examples/takeoff_and_land.py. Hopefully this information can help you resolve your issue.
-
@Eric-Katzfey Thank you for the reply,
If I understand correctly, the example you provided shows MAVSDK running directly on the VOXL — so on the drone itself. However, we want to run our script on a separate PC to be able to control all drones at once. We're not sure which port to use to communicate with PX4 while also taking advantage of the relocalization handled by VOXL.Thank you again for your help!
