@tahawaru please pull down the latest dev branch: https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-to-ros2/-/tree/dev?ref_type=heads
You can rebuild with the dev branch and see if you can both ros2 topic echo and rosbag record the encoded images. Please be aware that you will need to manually decode the images somehow on the receiving end (your application) as there isn't a built-in way in ros2/rviz to decode.
@jonathankampia, the way that voxl-microdds-agent works is that it wraps around the XRCE microdds agent binary from e-prosima, and bridges the px4 microdds topics to the DDS global data space (e.g. ROS2 topics).
The voxl-microdds-agent runs irrespective of any serialized message definitions, so it is just the transport. Probably not the issue, but if you still feel strongly that the agent is the issue, there are ways to test that.
Most likely, the mismatch is between px4_msgs and the voxl-px4 version. In that case serialized message definitions DO matter, and the mismatch could cause issues, but usually this is caught when the dds agent service runs (it will report errors you can view via systemctl status).
What sdk version are you using on the voxl2? You could checkout the submodule px4_msgs version that voxl-mpa-to-ros2 uses.
It would be difficult to troubleshoot without the actual offboard code to test, maybe post a snippet of your code?
For an offboard example using trajectory setpoints, you can also try this one which has been referenced a lot in the PX4 community
@tahawaru please pull down the latest dev branch: https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-to-ros2/-/tree/dev?ref_type=heads
You can rebuild with the dev branch and see if you can both ros2 topic echo and rosbag record the encoded images. Please be aware that you will need to manually decode the images somehow on the receiving end (your application) as there isn't a built-in way in ros2/rviz to decode.
@tahawaru there are a few intermediate steps to get ros2 offboard control:
disable offboard_mode (set to false) in voxl-vision-hub.conf line 237: "offboard_mode": "false"
Ensure voxl-microdds-agent is running. You can check by running systemctl status voxl-microdds-agent
Once these steps are complete, and you have the ros2 offboard fig 8 running, you should be able to echo the offboard control mode info either in an adb shell or mavlink console:
In adb you can use the command px4-listener offboard_control_mode which should return this
offboard_control_mode
timestamp: 314055050 (0.031338 seconds ago)
position: True
velocity: False
acceleration: False
attitude: False
body_rate: False
actuator: False
ros2 topic echo /fmu/in/trajectory_setpointManual mode is not really representative of the ability to switch into offboard mode and send setpoints. You will need to be able to arm in position or hold mode first. If you do not have an rc controller then try to arm in hold mode. If you are not able to arm in either mode, then there is a preflight check fail somewhere that needs to be addressed first.
Please verify the PC data in rviz2. What you see in rviz2 should look exactly alike to what is shown in voxal portal for tof_pc. Note that the point step size is 12. See this guide for more info on pointcloud data message data
Point cloud data will look very different from image data, since the data field is constructed very different from BGR image formats.
I have replicated these results in a starling2 as well, with voxl-mpa-to-ros2 dev branch:
Note that you may need to spin up a static transform to see the pointcloud data in ROS:
ros2 run tf2_ros static_transform_publisher 0 0 0 0 0 0 map world
@Captain-7th Are you able to arm and fly in position mode? According to the mavros state you are connected and the printout shows that the arming command is sent to the vehicle while in offboard, but it is possible there is a preflight check that is failing?
In the PerCameraMgr class, there is a function that process a single camera capture result (frame?) and prints the timestamp (meta.timestamp_ns).
We are attempting to compare the frame timestamp with the system time, which we have been able to do successfully, however it only looks like the camera frame is from hires small?
How can we distinguish between hires large and small in the hal3 camera manager code? We just need to compare timestamps but need to confirm if the metadata printed is from hires large or small, or both
Download
VERBOSE: Frame Timestamp: 1708331612528
Sys Timestamp: 1708373675076
Diff Timestamp: 42062548
VERBOSE: Gain: 54
VERBOSE: Exposure: 270328
VERBOSE: Received result from HAl3 for frame number 0
VERBOSE: Received 1 buffers from camera hires, partial result:0
VERBOSE: Received output buffer 0 from camera hires
VERBOSE: finished sending request for frame 5 for camera hires
VERBOSE: returning from SendOneCaptureRequest for frame 6 for camera hires
VERBOSE: hires procesing new buffer
VERBOSE: Camera: hires processing small vid frame
VERBOSE: added request for small video stream
VERBOSE: Sending request for frame 6 for camera hires for 1 streams
VERBOSE: Received result from HAl3 for frame number 4
VERBOSE: Received 0 buffers from camera hires, partial result:1
VERBOSE: Received result from HAl3 for frame number 1
VERBOSE: Received 0 buffers from camera hires, partial result:2
VERBOSE: Received metadata for frame 1 from camera hires
VERBOSE: Frame Timestamp: 1708398182893
Sys Timestamp: 1708435426587
Diff Timestamp: 37243694
VERBOSE: Gain: 54
VERBOSE: Exposure: 270328
VERBOSE: Received result from HAl3 for frame number 0
VERBOSE: Received 0 buffers from camera tof, partial result:2
VERBOSE: Received metadata for frame 0 from camera tof
VERBOSE: Frame Timestamp: 1708453167425
Sys Timestamp: 1708454078149
Diff Timestamp: 910724
VERBOSE: Gain: 0
VERBOSE: Exposure: 0
VERBOSE: Received result from HAl3 for frame number 1
VERBOSE: Received 1 buffers from camera hires, partial result:0
VERBOSE: Received output buffer 1 from camera hires
VERBOSE: finished sending request for frame 6 for camera hires
VERBOSE: returning from SendOneCaptureRequest for frame 7 for camera hires
VERBOSE: added request for small video stream
I am looking to integrate a neopixel (WS2815) on the VOXL2, but was concerned that the SPI gpio can not be re-purposed for general purpose GPIO. There is a voxl2 linux guide that shows a gpi-mod kernel driver so there is concern that we would not be able to access the GPIO to faciliate the neopixel driver signal in user space (like how RPI.GPIO python library works on a raspberry pi)
@Captain-7th Can you do a rostopic echo of
/mavros/state
You should be able to see info printed if the autopilot is connected or disconnected.
We are extensively testing camera failure cases on OpenVINS using the cvp/VFT instead of the default 'vins' tracker.
The big issue for is is that there are some flyaways still occuring in nominal conditions even when using 2-camera tracking (non-stereo). More importantly there doesn't seem to be true redundancy if one camera fails. It makes sense in a stereo configuration if one camera fails and tracking no longer works, but in the cvp multi-camera (front and downward) setup, shouldn't the other camera still track features and hold position?
Is there a timeline for when VFT will be stable, and is it actively being worked for a stable release in the near future?
One thing that helped with qvio feature tracking was the ability to use voxl-logger, but this functionality doesn't exist yet for openvins, hence using the voxl portal vio page to see when tracking fails
What hardware are you using? Starling v1 with upgraded motors
What version of software are you using? See below
How have you configured the software? See below
Do you have any logs? No logs, but vio page showing a flyaway in nominal conditions:
Can you share pictures of your setup? Front and down camera (front and down are ar0144 on my colleague's setup, but i have an ov7251 front and ar0144 down
Have you looked at the source code? Yes, looked at voxl-feature-tracker on gitlab and there are some apparent bugs in the software or it is not finished
--------------------------------------------------------------------------------
hw platform: M0054
mach.var: 1.0.0
--------------------------------------------------------------------------------
voxl-suite: 1.2.0
}, {
"enable": true,
"group_name": "tracking_FDR",
"output_pipe": "tracking_feats",
"overlay_pipe": "tracking_feat_overlay",
"group_cams": [{
"enable": true,
"input_pipe": "tracking_front",
"tracker_type": "cvp",
"num_features": 30
}, {
"enable": true,
"input_pipe": "tracking_down",
"tracker_type": "cvp",
"num_features": 30
}, {
"enable": false,
"input_pipe": "tracking_rear",
"tracker_type": "cvp",
"num_features": 30
}]
}, {