Which depth sensor to use for optimal voxl-mapper performance
-
Hello everyone,
I have a VOXL Flight Deck (apq8096) with hires, tracking, and stereo cameras only. The VOXL is running a 4.0.0 system-image, voxl-suite-0.9.5, and sdk-0.9, along with dfs-server-0.3.1 and voxl-mapper-0.1.5 (the version that supports sdk-0.9 with vvpx4 instead of vvhub).
The voxl-mapper config file is set to take depth data from the dfs_point_cloud pipe instead of tof since tof is not available on the flight deck:
voxl:~$ cat /etc/modalai/voxl-mapper.conf /** * This file contains configuration that's specific to voxl-mapper. * * RRT Parameters * rrt_min_distance: Minimum edge distance in the RRT. Should be larger than voxel size otherwise * there will be wasted computation. Default: 0.2 * rrt_max_runtime_nanoseconds: Maximum amount of time allowed for the RRT planner to run. Default: 1000000000 * rrt_goal_threshold: Minimum distance to goal before a point is considered to be at the goal. Default: 0.05m * rrt_prune_iterations: Number of iterations to run pruning on the RRT path for. Default: 100 * rrt_send_tree: Whether to send the RRT tree for visualization in portal. Default: false * treat_unknown_as_occupied: Whether to treat unknown cells as occupied. Default: true * * Smoother Parameters * loco_resample_trajectory: If true will take the initial guess from the linear solver and resample the * trajectory to get a new trajectory with num_segments in it that is then * passed to the nonlinear solver. Default: true * loco_resample_visibility: If true will resample before running the linear solver. Uses the visiblity * graph and a time estimation of the entire path to resample points. the * visibility graph is essentially the graph made up of the waypoints passed to * the smoother. Default: false * loco_num_segments: The number of segments that the resampled trajectory will have (only applies * if resample_trajectory is true. Default: 5 * loco_add_waypoints: Adds waypoints into the nonlinear smoother to optimize passing through each * waypoint. If disabled then waypoint cost weight has no effect. Default: false * loco_scale_time: Scales the segment times evenly to ensure that the trajectory is feasible * given the provided v_max and a_max. Does not change the shape of the trajectory, * and only increases segment times. Default: true * loco_optimize_time: Runs an additional optimization step (using nlopt) to optimize the segment * times in order to better meet the dynamic constraints. Default: true * loco_split_at_collisions: Adds additional points to the trajectory if any portion of the initial linear * solvers trajectory is in collision. Default: true * loco_v_max: Max velocity of robot. Default: 1.0 * loco_a_max: Max acceleration of robot. Default: 2.0 * loco_yaw_rate_max: Max yaw rate of robot. Default: PI/4 * loco_sampling_dt: Time step delta at which to sample points from the trajectory to check for collisions * this is used in the linear solver and generally only when split at collisions is true * Default: 0.1 * loco_smoothness_cost_weight: Weighting for smoothness of derivative we are optimizing for. Default: 0.1 * loco_collision_cost_weight: Weighting for collisions. Default: 10.0 * loco_waypoint_cost_weight: Weighting for waypoints (has no effect if add_waypoints_ is false). Default: 1.0 * loco_min_collision_sampling_dist: Time step delta at which to evaluate cost/gradient for collisions. Default: 0.1 * loco_verbose: Whether to print debug statements or not. Default: false */ { "tof_pipe_0": "/run/mpa/tof", "tof_0_enable": false, "tof_0_rate": 10, "tof_extrinsics_0_name": "tof", "tof_1_pipe": "/run/mpa/tof", "tof_1_enable": false, "tof_1_rate": 10, "tof_extrinsics_1_name": "tof", "tof_2_pipe": "/run/mpa/tof", "tof_2_enable": false, "tof_2_rate": 10, "tof_extrinsics_2_name": "tof", "tof_3_pipe": "/run/mpa/tof", "tof_3_enable": false, "tof_3_rate": 10, "tof_extrinsics_3_name": "tof", "depth_pipe_0": "/run/mpa/dfs_point_cloud", "depth_pipe_0_enable": true, "extrinsics0_name": "stereo_l", "depth0_rate": 5, "depth_pipe_1": "/run/mpa/voa_pc_out", "depth_pipe_1_enable": false, "extrinsics1_name": "stereo_l", "depth1_rate": 5, "depth_pipe_2": "/run/mpa/tflite", "depth_pipe_2_enable": false, "extrinsics2_name": "stereo_l", "depth2_rate": 5, "depth_pipe_3": "/run/mpa/dfs_point_cloud", "depth_pipe_3_enable": false, "extrinsics3_name": "stereo_l", "depth3_rate": 10, "robot_radius": 0.51, "voxel_size": 0.4, "voxels_per_side": 16, "esdf_save_path": "/data/voxl-mapper/esdf_map", "tsdf_save_path": "/data/voxl-mapper/tsdf_map", "mesh_save_path": "/data/voxl-mapper/mesh", "esdf_max_distance": 2, "esdf_min_distance": 0.10000000149011612, "esdf_default_distance": 2, "esdf_inner_sphere_radius": 0.20000000298023224, "esdf_outer_sphere_radius": 0.600000023841858, "rrt_min_distance": 0.20000000298023224, "rrt_max_runtime_nanoseconds": 1000000000, "rrt_goal_threshold": 0.05000000074505806, "rrt_prune_iterations": 100, "rrt_send_tree": false, "treat_unknown_as_occupied": true, "loco_num_segments": 10, "loco_smoothness_cost_weight": 0.1, "loco_collision_cost_weight": 10, "loco_waypoint_cost_weight": 1, "loco_min_collision_sampling_dist": 0.1, "loco_add_waypoints": false, "loco_scale_time": true, "loco_split_at_collisions": true, "loco_resample_trajectory": true, "loco_resample_visibility": false, "loco_verbose": false, "loco_optimize_time": true, "loco_v_max": 1, "loco_a_max": 2, "loco_yaw_rate_max": 0.78539816339744828, "loco_sampling_dt": 0.1 }In doing so, I was able to get some results from running the voxl-mapper, but they were noisy and missing big portion of the room that were mainly walls and flat surfaces. I assume this has to do with the fact that dfs has no capability to tell the depth of flat surfaces since there are no features to compare from the stereo images.
Q1- Is it possible to fuse other data with the dfs_disparity image, like LiDAR or laser sensors, to generate a better point cloud that gets fed to voxl-mapper?
Q2- Is it possible to use the tflite's 'depth-perception' algorithm to generate another point cloud data and fuse it with the dfs_point_cloud data? If yes, then is the fusion done by enabling two pipes in the voxl-mapper.conf file, or by fusing them into one pipe first and then feed it into voxl-mapper?
Q3- Is the VOXL tof camera fully supported on this platform? and if I use it on its own will it resolve this issue entirely?
Thanks
-
@Jasmi The TOF sensor is better for indoors, stereo is better for outdoors
-
@Moderator Thank you for your response.
For the Tof sensor, according to the information found here, it says that I should connect it to the J2 connector on the flight deck. I assume this means that I have to replace the hires camera with the tof camera. If I do so, can I use the same extrinsics and intrinsics calibration of the hires camera for the tof camera? Or do I have to re-calibrate it?
