Posts made by Cliff Wong

Hi there, if you're using GPS+mag+vio then you'll need the following PX4 params changes in addition to our vio_gps_baro.params helper file:

EKF2_MAG_TYPE = automatic (voxl default)
EKF2_MAG_CHECK = 1 [or 2/3 for more sensitivity] (voxl default is 0)

Having mag_check = 1 will activate EKF2's field strength checker such that as you proceed indoors should stop mag fusion completely when the field strength goes outside thresholds (hardcoded within PX4 ~ .4gauss variance). It's similar to GPS's DOP threshold where we expect both (gps and mag) to exceed the thresholds where PX4 will stop fusion of those modalities and eventually use vio solely.

PX4's logic tries to maintain "smooth flight", so expect some delay/drift as it starts turning on/off the above (i.e. it will not be instant). Hence, adjust performance in those transitions, you'll need to modify ekf2's baro, gps, and mag *_GATE params accordingly.

Note PX4 will adopt whatever heading is being derived from the activated sensors (GPS/Mag/vio) to construction its "global frame", so proceeding indoors you will still be operating under a global frame coordinate system (will still have a north reference based on last good mag/gps check).

Hey there, voxl-logger for openvins is only avaiable using a combination of raw flags, e.g. -c tracking_front -i imu_apps -r ov -r ov_extended -r ov_overlay Yes the -o option for odometry is not completed yet for openvins. That hsould get you the log data for openvins

As for simulating camera stream failure, it's very likely any attempt to cover a camera will increase the exposure of that stream, and in some cases you will still see random features: for example, covering with a finger will still pick up features from your finger print. We're still actively improving voxl-feature-tracker and should have a new version available soon.

@Jetson-Nano If using QGC for mission planning, indoor missions are more of an approximation since openstreetmap displays no indoor spaces (e.g. approximating waypoints on the roofline of buildings). If there's a way to overlay an indoor floorplan/raster or add custom maps using a WMS server into QGC would be ideal. Then you can ingest a floor plan of a indoor space and perform precise waypoint planning. Note QGC mission planning provides distance info between waypoints, so one can construct an fairly accurate flight pattern (e.g. for navigating known aisleway & starting point for example).

To describe what's happening under the hood: we're using QGC's takeoff mission item as the local position origin for VIO, and its registered lat-lon is used as a global position offset (though you can manually type it in as well in QGC). From there, waypoints are executed in reference to that origin and true north (of the map). In the end, you're basically flying a pattern based on the take off point and true north. Hence, it doesn't matter where you fly on the map cause this feature is telling PX4 where the drone is located in global space (when running vio only)--and executes the same flight pattern [on the map] from the distances & heading.

Alternatives for indoor "mission planning": we do offer voxl-mapper (trajectory-style) as well as wps mode in voxl-vision-hub (waypoint-style), though both are offboard mode planners in local space and thus precise by setup, but being offboard-mode based, you do not get all the safety measures that PX4 mission mode provides (e.g. RTH, PL, Geofencing, Auto-mode, etc...).

@Jetson-Nano Hi there, here's a tutorial on using this feature.

The video applies one of the vins_vio params-- based on if you are planning indoor or outdoor missions in QGC. The drone will automatically acquire heading from the internal compass. Note, the compass is not used for position estimation, just to acquire vehicle heading at power up. This is required in order to assign a global heading "at origin" (aka takeoff) since we are dealing with QGC's global Lat Lon system for mission planning. This allows a saved QGC-GPS mission be reused as long as a takeoff position is added....and loaded and executable under vio-only position estimation of course.

As an alternative, one may not use the above params, but instead use the option of manually entering in the heading of the vehicle. That is done in the takeoff item's heading field of a QGC mission. That is to allow manually heading change on the fly.

Hi there, yes, unfortunately the repo for OpenVINS is still under construction as it's under active development by the ModalAI team. We'll have it available soon

voxl-open-vins-server does a similar process to the ROS implementation when using camera data--in order to sync frames, data comes in a stereo-type packet. During camera calibration, instead of Kalibr, we are using our own voxl-calibrate-camera implementation that comes with the SDK. And as needed, one can turn off the stereo output in voxl-camera-server and calibrate each camera individually. That would be best on the ROS side as well where you can use split the images from camera-server like this before publishing to Kalibr (where 2 topics work best for openvins), then calibrate independently.

You could try voxl-calibrate-camera instead and yes it will output opencv-format intrinsics where you can copy the needed values into the ROS launch file. We create extrinsics using mechanical design. In both cases, OpenVINS will refine them at runtime during the first few seconds of operation.

You can upgrade to the latest dev version via editing your /etc/apt/sources.list.d/modalai.list on voxl2 to point to the dev repo. Then run apt update && apt upgrade voxl-open-vins-server. Looking at the /etc/modalai/voxl-open-server.conf may give you some ideas on how the roslaunch file can be setup as well.

@Jetson-Nano
Hey there, yes, if you are getting the QGC message that the "system is not ready" and will not let you arm in position mode, then you need a patch that sends the correct heartbeat message in PX4 to allow arming. It is in the SDK1.1.1 release, So I believe an apt-get upgrade should solve your problem.

But if needed the code is described here.

Hi all, couple things to clear up on this feature:

With the above parameters you are both setting limits for obstacle avoidance (OA) and collision prevention (CP)--2 different features. Details of each are here: OA and CP

To enable obstacle avoidance, make sure these parameters are being used.

Now collision prevention takes the additional step where you need to set param COM_OBS_AVOID=1. If you get a "Avoidance system not ready" message then that is a know s/w bug that is will be available on our development updates repo shortly. Also note these modes only work when MPC_POS_MODE is 0.

Lastly to disable OA, use these parameters.

Hi there, do you have COM_OBS_AVOID=1 (enabled)? You can set the parameter directly (params) or there's a menu option in QGC.

If so, can you arm the drone? With *_AVOID=1 I suspect all will work as you are seeing obstacle_distance data on mavlink.
If not, then you should get a QGC report "Avoidance system not ready"--let us know which case you end up with.

Hi there, looks like there's some confusion with what mode is being tested. Here's some clarification. PX4 has 2 approaches to handling obstacle distance data:
a. Obstacle Avoidance(OA) feature which stops the aircraft if object detection occurs from the voxl-voa service. This requires CP_* parameters set that you can compare with your drone listed here based on your application.

b. Collision Prevention(CP) feature, which not only includes OA, but also rerouting capability in all autonomous modes (position, alt, mission, offboard) and leverages the same voxl-voa service. This requires not only the CP_* parameters but also requires COM_OBS_AVOID = enable.

If your application requires (a), you can leave com_obs_avoid = disable and OA should work out of the box as is. If your application requires (b), then we have discovered a software bug in the VOA service. And the main reason you get the "collision module not ready", no arm possible. It has been corrected [code here] and binaries will be available in our gitlab daily builds shortly.

@AP123 Hey there. Are you running VOA against a stereo camera pair or via a ToF sensor?

If you run voxl-inspect-points voa_pc_out and see data, then the voxl2 SDK services are good and generating the correct mavlink obstacle distance points to PX4. If not then there's a configuration issue.

If that is good, then you need to make sure PX4 has
MPC_POS_MODE = Simple pos control or smooth pos control, COM_OBS_AVOID is enabled, CP_DIST > 0 (e.g. 2) and you must have an established home position (or good position from qvio if that is the case). Typically an "obstacle system not ready" is due to the MPC_POS_MODE (I think we default to acceleration-based input).

If all those check out, then it's likely missing a home position requirement in PX4 & not voxl, and we'll need to look that up.

Hey there, if you are flying solely with VIO (no GPS), suggest loading our indoor estimator params to add to the current setup on PX4. Outdoor vio+gps is here. These set the vio sensitivity that is in line with the suggestions mention on your PX4 forum's thread. Also we're assuming that extrinsics.conf values have not changed and are same as the factory settings.

Secondly, if you run voxl-inspect-pose vvhub_body_wrt_local on voxl2 it will apply the extrinsics and output pose that should be inline w/PX4's local position. If those values are correct, then we're looking at a PX4 tuning/config issue (we can still help).

One thing that looks weird on your PX4 flight log is Roll and Yaw rates are not keeping up w/respective setpoints (noisy). [Hence, thirdly, suggest an] IMU calibration on the PX4 side.

Lastly in /etc/modalai/voxl-vision-hub.conf there's a parameter called en_reset_vio_if_initialized_inverted. It is set to true as it's to account if vio was accidentally initialized upside down (while carrying the drone for example). You can set it to false and if there's a mismatch on VIO-PX4, you'll see it immediately.

Yes, please disable the qvio-server when running non-evdata modes. Note, you will have an issue when AID_MASK = 0 and you try to go into position/offboard/auto modes due to you will be in dead reckon mode and may not have an established (or correct to ground truth) NED origin.

Modules such as Precision Landing in PX4 perform a indirect correction using relative camera coordinates translated to the global frame upon identification if the "tag/marker". I suggest you take a look into the Precision Landing module of px4 as an example in solving your problem. Of course that is using the IR markers available out there. Now we have internally modified that module for april tags in the landing case as well that will be available shortly.

This method corrects the drone relative to actual ground truth (i..e the ARtag/IRTag), but does not correct the Global Frame position itself as its used as a relative offset to correct its setpoint position. That offset could be used to correct the global frame position itself (doesn't necessarily exist in PX4), but then again it would be better to transmit the actual GPS position of the tag to account for the curvature of the Earth, etc... and then calculate a proper offset.

Looking at your mavlink log on estimator_visual_odometry_aligned, I can confirm VIO resetting to 0 at the moment your local position resets (picture #1).

(pic 1)

On a rover typically VIO will reset on 3 situations (everything else working normally as specified):

1. Fast yaw transitions (I do see a large manual input/actuator output around that time--indicated on pic #2). To solve this, have your vehicle turn/tuned slower on yaw motions.
2. auto-exposure not keeping up with rapid lighting conditions (high noon sunlight + tree shadows), the
3. too much vehicle vibration (e.g. treads).

From the logs it appears you have a max yaw/roll input into the rover, that indicates scenario #1 above. If you slow down you yaw movements may solve the problem, please try that 1st (there are other, more complicated options).

(pic 2)

What you can do aside from following the vio troubleshooting guide is to monitor the vio stream on QGC. In order to do that, you need to edit the voxl-streamer service file on VOXL. Basically:

1. login to VOXL
2. edit /etc/systemd/system/voxl-streamer.service
3. on the Exec line that says /usr/bin/voxl-streamer, append to the end of it "-c qvio-overlay".
4. cycle power and QGC video should show the VIO debug video (and not the high resolution stream), showing at the top: vio xyz position and number of features
5. Run thru your test again to recreate the error, when it occurs, keep in mind the number of features, if it drops under 5, you'll see vio reset confirming the reset is due to one of 3 scenarios above. If not, post a video of that as it would be a configuration issue & we'd need to see the video stream on what's happening to debug further. Good luck.

Easiest is to get ros-base installed to run mavros. To install:

Install ros-melodic-ros-base via http://wiki.ros.org/melodic/Installation/Ubuntu
(-desktop-full should work too, but to be safe, based on system image version you may have, to avoid any dependency issues).

execute: sudo apt-get install ros-melodic-mavros
execute: sudo apt-get install ros-melodic-tf2-geometry-msgs ros-melodic-tf2-sensor-msgs
execute: sudo geographiclib-get-geoids egm96-5
execute: /opt/ros/melodic/lib/mavros/install_geographiclib_datasets.sh
execute sudo apt-get install ros-melodic-stereo-image-proc ros-melodic-navigation

ROS and RB5 work the same as other PX4/VOXL products: that it leverages PX4 offboard mode to send mavros command/data. An example ROS (to px4 via mavros) application can be found here that you can drop into your ros workspace.

Good to hear voxl services are ok, that leads to PX4's ekf2_aid_mask. It should be 280, aka all the external vision inputs are on. Then you should see xy odom in QGC's mavlink inspector.

Also after resetting vio-server, you can run voxl-inspect-qvio to monitor valid localization information. If that is good, it's very likely (since you're getting z telemetry) you need to activate external vision solution in px4:
https://docs.modalai.com/flying-with-vio/

Look at the ekf2_aid_mask and substitute any flight core reference for the cube. Likely your aid mask is 1, which is gps and px4 will not report odometry mav data (z is fusing baro+vision and why you get z and no xy).

Hi there, Yes, let's check the camera stream:

on your host computer, in a terminal, run: "xhost +"

power up rb5, ssh login: "ssh root@10.8.0.x -X" (make sure you have the -X option)
In the rb5 ssh session, run the following:

systemctl stop rb5-streamer   

Then run in the ssh session:

gst-launch-1.0 -v qtiqmmfsrc video_0::bitrate=100000 camera=3 ! video/x-raw,format=NV12,width=640,height=480,framerate=15/1 ! videoconvert ! xvimagesink 

If you get no video, then we have a camera/streamer issue (unlikely). Power cycle and confirm again that there's no video.

But If you get video, then disable rb5-streamer via:

rb5-streamer-configure (choose '2')

power cycle rb5, again, ssh back in and manually run the streamer via:

rb5-streamer -d -c 3

and try to connect via vlc/qgc, the debug output will show a connection of 1 when connected. If not, then there should be output on the issues, If there is no debug output (and stays '0') then there is a network restriction (e.g. bandwidth is too little) issue. Re enable the streamer by running rb5-streamer-configure (choose '1').

Hi Sergio. I'm handle a number of rover configurations here for the team. Reading thru the posts, let's take a step back to confirm your setup.

a. you are using the cube to output IMU data into voxl-cam.
b. you are using voxl-cam to output odom data [back] to the cube for (e.g. external vision localization)
c. the camera has been moved from the stock position of -45deg from horizon w.r.t. voxl-cam body to 0deg from horizon w.r.t. voxl cam body.
d. the cube is mounted w/IMU in standard orientation (0 deg to horizon/forward, aka imu RPY is 0,0,0)
e. you are not using a modalai flight core (option for voxl-cam purchase)

First of make sure your PX4 SENS_BOARD_ROT (or AHRS_ORIENTATION if Ardupilot) is set for a cube setup (I recall should be "No rotation" as well as from your photo). Please reply if that is all correct. If so, then we can move to the next step where I can show you how confirm vio's pose estimation (after that then we'll look at position accuracy).

Mavlink messages for setting flight mode use the MAVLINK_MSG_ID_SET_MODE message type (same as APM).

Modes are defined in the uOrb layer under the vehicle_status.msg fie, which C headers are auto-generated at build time. The auto-generated file is under the build directory as vehicle_status.h via the NAVIGATION_STATE_* variables.

And example of using modes in runtime code are in the mavlink_messages.cpp file
(search for vehicle_status_s::NAVIGATION_STATE_AUTO_MISSION) and modes are processed in the Commander.cpp file. Both files under the src/modules directory of PX4.