ModalAI Forum

Re: Using Sentinel Drone with IntelRealSense Depth Camera D435

So this isn't a problem. I have seen on the forum some talk on using the D435i with the Voxl2 through the USB connection and can confidently say we succeeeded, so we are linking the Github method to get it working. This was with SDK 1.1.1. We also used the USBA port into a USB hub bought of the shelf to connect both wifi and D435i into the hub in case anyone wants to try using the D435i our their voxl2.

For our stereo application we found the D435i gave signficantly bettere stereo depth images than the native stereo-dfs-server, but it is up to you guys which one you prefer.

https://github.com/IntelRealSense/librealsense/issues/12442#issuecomment-2192775535

At the end of this you should be able to use ros2 run rqt_image_view rqt_image_view in order to see the output after running the ros2 realsense camera node

@Suvasis-Mukherjee

We have succesfully integrated the D435i to stream video recently with the Voxl2 through ROS2. however this process is a bit involved with a few twists and turns, so it is possible

@Eric-Katzfey thanks for the quick response. Yeah personall we ended up mounting the GPS further up with a pole in a custom design. Do you guys have a link to the px4 parameters you use with the barometer onboard the Voxl2 or does it need some sort of calibration?

Hi Modalai,

Recently we tried flying with the native barometer/recommended 3DR GPS on the website and found the performance was quite bad comapred to other products

Assembled in USA | Autopilots that Accelerate Drone and Robot Autonomy

ModalAI develops sUAS, drone and robotics autonomous computer vision, flight control and communications systems, and manufactures in the USA. ModalAI's VOXL platform advances Qualcomm Flight, combining ROS and PX4 for Obstacle Avoidance and GPS-Denied indoor navigation, in ready for development to upgrade your drone.

ModalAI, Inc. (www.modalai.com)

Do you have any suggestions on how to improve them or an alternative product to do no Vision based height estimation

@Alex-Kushleyev hi Alex thank you so much for the help. We figured it out after updating some of the firmware. It flies much smoother with the jeti RC now

Oh glad to hear something from your end.

Currently we are using VoxlESC and the Px4 from SDK 1.1.1. Recently I flashed the board but the remote control is no longer detected. Also how do we set the parameter of the voxl2_io_func8 is this in the start up /usr/bin/voxl-px4-start?

Thanks again for the help,

Jeff

@Alex-Kushleyev We have the current thing

Attempting to start M0065 SBUS RC driver for original M0065 FW
INFO [muorb] SLPI: qshell gotten: dsp_sbus start
INFO [muorb] SLPI: arg0 = 'dsp_sbus'
INFO [muorb] SLPI: arg1 = 'start'
INFO [qshell] Send cmd: 'dsp_sbus start'
ERROR [muorb] SLPI: Failed to execute command: dsp_sbus start
Starting M0065 SBUS RC driver for original M0065 FW failed
Attempting to start M0065 SBUS RC driver for new M0065 FW
INFO [qshell] Send cmd: 'voxl2_io start -d -p 7'
INFO [muorb] SLPI: Ok executing command: voxl2_io start -d -p 7

We will try updating the firmware and let you know how ti goes based on what you say

@Alex-Kushleyev Quick question how do we check the M0065 Version?

Also have you guys tried Futaba remotes are they OK? Just wondering since they are on the list of RC with Graupner, but it seems Graupner's remotes are sold out everywhere
https://docs.modalai.com/voxl2-rc-configs/

Hi Modalai,

Recently we connected a Jeti receiver to our voxl2 board using Sbus through the voxlio board with the px4 running on the voxl2 and ESC connected to the main Voxl2 board

We noticed there is a significant lag between the remote’s command and the reaction from the Voxl2. The throttle/pitch/roll/arm/killswitch on our remote lags behind the real action. This problem does NOT exist if we directly plug the spectrum receicer to our voxl2 board, so this is NOT a control parameter issues

We were wondering if you noticed this issue when using other remotes like Graupner sbus to the voxl2 board?

Thanks,

Jeff

@Moderator Right I think that is a bit too difficult.

I just edited the voxl-mapper library to send the pointcloud and it works now to get the map in ROS2 with the mpa_ros2 interface. It is a bit weird that you guys have this esdf publish pipe as a pointcloud but it doesn't seem exposed naturally

server/voxl-mapper/voxl_mapper.cc · master · voxl / VOXL SDK / Services / voxl-mapper · GitLab

GitLab.com

GitLab (gitlab.com)

void TsdfServer::publishEsdfPointcloud(){
    std::vector<point_xyz_i> esdf_ptcloud;

    voxblox::Layer<voxblox::EsdfVoxel> *layer_ptr = esdf_map_->getEsdfLayerPtr();
    createDistancePointcloudFromEsdfLayer(*layer_ptr, esdf_ptcloud);

    // intensity ptcloud for distance
    point_cloud_metadata_t esdf_meta;
    esdf_meta.magic_number = POINT_CLOUD_MAGIC_NUMBER;
    esdf_meta.timestamp_ns = rc_nanos_monotonic_time();
    esdf_meta.n_points = esdf_ptcloud.size();
    esdf_meta.format = POINT_CLOUD_FORMAT_FLOAT_XYZC;

    pipe_server_write_point_cloud(COSTMAP_CH, esdf_meta, esdf_ptcloud.data());
}

Hi Modalai,

Currently, I have the voxl mapper running, but when I run voxl_mpa_to_ros2. I see that only voxl_mapper_aligned_ptcloud is brought over to ROS2 because the interface for voxl_mpa_to_ros2 only has a pointcloud interface. What sort of interface would be needed to be made for the voxl_mapper_costmap to ROS2 or is there a certain guide. Also would it be possible to expose either the TSDF/ESDF from voxl-mapper to the MPA and then bring it to ROS2?

voxl_mapper_aligned_ptcloud
voxl_mapper_costmap
voxl_mesh
voxl_mesh_endpoint

Thanks,

Jeff

Nevermind I figured it out. Once the voxl_mapper service is running it is automatically configured to run the voxl_mpa_ros2 as a pointcloud

@tom We have experimented with it, but I am not sure about the format of voxl-mapper, and have trouble getting voxl-mapper working with just stereo cameras. Do you guys have more resources on it besides the github page?

We are also wondering if it is worth it getting the voxl-mapper working, or will it automatically work with voxl_mpa_to_ros2 pointcloud interface or have some different representation which requires us writing our own piep interface?

Also what branch is the stable version of voxl_mpa_to_ros2 because I can see the master branch doesn't allow us to stream QVIO odometry?

colcon_ws/src/voxl_mpa_to_ros2/src/interfaces/qvio_interface.cpp · master · voxl / VOXL SDK / Utilities / voxl-mpa-to-ros2 · GitLab

GitLab.com

GitLab (gitlab.com)

I believe you are publishing the same topic 2 different message types which I don't believe is allows in ROS2.

void QVIOInterface::AdvertiseTopics(){

char topicName[64];

sprintf(topicName, "%s", m_pipeName);
pose_pub_ = m_rosNodeHandle->create_publisher<geometry_msgs::msg::PoseStamped>
    (topicName, rclcpp::SensorDataQoS());

odom_pub_ = m_rosNodeHandle->create_publisher<nav_msgs::msg::Odometry>
    (topicName, rclcpp::SensorDataQoS());

m_state = ST_AD;

}

Does there exist any tools to pipe the output of Voxl-mapper to a topic in ROS2? Also is it possible to use voxl-mapper with just stereo cameras generating pointclouds or is a ToF sensor needed

@Alex-Kushleyev Sorry for the confusion I mean Disparity Image.

On further examination, I think this is a network problem. When streaming high bandwith objects like Disparity images or images in general across wifi. I think effectively managing the bandwidth can alleviate this problem.

@Alex-Kushleyev

Thanks for the information. I think I understand what is going on my end then.

This happens if we leave the pipe on while streaming depth_images from stereo_disparity for a minute or two. After which it happens very frequently.

Currently, I am using the voxl_mpa_to_ros2 repository. Do you have any idea what causes the following error

ERROR: invalid metadata, magic number=0, expected 1448040524
most likely client fell behind and pipe overflowed

This only happens when I try streaming the depth_image through the piep> There is no problem with lowbandwith data like imu_app/qvio odometry. Do you know what the root cause is for this or if there is a way to solve this in ROS2?

Thanks,

Jeff

Quick question I was looking through VOXL2 DFS Server. Do you know what the camera_information or the formula to convert the pixel values of the stereo_disparity image to a pointcloud for further understanding?

Traditionally, I believe the formula should be something like. However I am not sure what the equivalent fx,fy,cx,cy would be in the stereo image.

for (int i = 0; i<depth_image.rows; i+=2)
{
    for (int j = 0; j < depth_image.cols; j+=2)
    {
      geometry_msgs::Point32 pt;
      pt.z = depth_image.at<float>(i, j); 
      pt.x = (double) ((j - cx) * pt.z / fx);
      pt.y = (float) ((i - cy) * pt.z / fy);
      cloud_.points.push_back(pt);
    }
}

Hi Quick question for the Voxl2 Mapper is there a way to get occupancy or TSDF/ESDF into ROS/ROS2 topic publishing?

Also if I want to run the voxl2_mapper with the voxl-dfs-server and no time of flight? How can I check the pipes or what is the output pipe of the voxl-dfs-server?

"depth_pipe_1":	"/run/mpa/stereo_front_pc",
"depth_pipe_1_enable":	false,
"extrinsics1_name":	"stereo_front_l",
"depth1_rate":	10,
"depth_pipe_2":	"/run/mpa/stereo_rear_pc",
"depth_pipe_2_enable":	false,
"extrinsics2_name":	"stereo_rear_l",
"depth2_rate":	10,
"depth_pipe_3":	"/run/mpa/dfs_point_cloud",