Posts made by Cliff Wong

option 2: upgrade to SDK 1.6 as we have a new version of VIO and numerous bug fixes

Directions are found here. where the SDK for starling 2 is found here.

Hi there, looking at the files and review our analytics here since you were flying fine in position mode that says VIO was likely ok (as per you config files).

We did have a some changes between vision-hub and px4 that make have been incompatible with the 1.4.5 version. Hence you have 2 options:

option 1: debug & confirm either probem lies in voxl-vision-hub or PX4.
(a) you need to ssh onto the vehicle, in the remote terminal, run:
voxl-vision-hub --debug_offboard

That will display streaming position data, which is the traj commands to PX4. Then you may restart/start voxl-mapper (e.g. systemctl restart voxl-mapper). In the vision-hub streaming output, you'll see a print out like Trajectory Monitor connected to voa pointcloud.

(b) you can now switch to position mode, fly around in that mode to map as normal, then stop to hover. Draw your simple trajectory in portal and execute. Ensure the draw trajectory is within the space using the mouse and rotating around the area--portal can sometimes make the 'plan point' appear close to the drone, but it is actually very far away so you need to check at different viewing angles. When you execute the plan, the streaming output in the vision-hub display above should reflect the drawn trajectory. In both your example and mine, the Z position output should stay relatively close to the current. If it rises, we have a trajectory calculation problem (drone is doing what it's told). If it rises while the position data is fairly minimal, then PX4 likely has failed (VIO or dropped out of offboard mode). Please try this and report back. We suspect VIO may have failed. Here's a example of the output:

@greg_s Hi there, can you post these 2 files from your drone:

/etc/modalai/voxl-vision-hub.conf
/etc/modalai/voxl-open-vins-server.conf

One thing from your screenshot is I sort of see the drone's position and the trajectory planned. For example as in this screenshot, the drone's xyz position should be visible (xyz axes) and a trajectory (black & red lines) should be displayed before executing a trajectory. The drone should be moving in realtime hovering in position. If it's static the vio was likely lost.

@berayksl Hi there, to address your questions:

• The inconsistent VIO performance between flights

appears the inconsistent vio behavior on multiple arm/disarms is due to the way voxl-open-vins-server re-initializes after a disarm event. Are you still using in the voxl-open-vins-server.conf option "en_vio_always_on": false and "en_auto_reset": true?

If so, the next thing that can be an issue is takeoff speed. To ensure VIO has the correct origin (0,0,0) we have a conf parameter called: "takeoff_accel_threshold": 0.5, This is essential a takeoff detector and is set for a aggressive take off (>80% throttle)

If you are taking off very slow and gentle, it's possible the detector is not being tripped. Try lowering it from 0.5 to 0.3.

• The “Yaw estimate error” message (should I be concerned if QGC shows no error)?

If the error doesn't repeat, it's just VIO telling PX4 to reset it's heading and PX4 complains on reset. This can be ignored. BUT if QGC keeps repeating the announcement, you have a PX4 yaw_gate issue--make sure in PX4 the param EKF2_EV_CTRL has the 'yaw' option checked.

And as a sanity check cause you are seeing an R/C issue: goto the radio tab in QGC and confirm your pitch/roll/yaw inputs are calibrated and centered such that you are not sending biased commands.

• The drone ignoring RC inputs during the second flight

Are you getting a r/c "manual control lost" announcement in QGC or just lack of control when trying to move the drone? On the former case we have notice users in environments with lots of metal (e.g. warehouse) will have RF interference issues.

At this point, we have numerous improvements in SDK 1.5.0 for Max and it maybe worth the upgrade if your system design allows it. It has improvements with the vio-always-on issues, vio resets, and some PX4 altitude improvements.

@berayksl Hi there,

In the VOXL2 file /etc/modalai/vio_cams.conf all the cameras should be listed as a JSON record. the top-level disable attribute of each record can be edited to turn cameras on and off (make front true and the others false for just the front camera).

On calibration, instructions can be found here.

If the camera's physical positions changed from delivered, then you'll need to update the extrinsics.

@berayksl
Hi there,

1. Since you were able to run a limited bench test, having it fly away after a couple movements indicates 3 possibilities:

• extrinsics are incorrect -- either bad imu_apps to cam# values or camera outputs are flipped.
• intrinsics are incorrect (bad camera calibrations)

2. yes you can run without mag enabled. Should work normally if (of course) VIO is running normally.

3. As for PX4 parameters, VIO enabling can be setup applying this param file.

My thinking is the camera extrinsics maybe off or mismatched to the camera ID. If you can only enabling one camera at a time and run the bench/hand test as if like a single camera vio setup, hopefully it will reveal the misconfigured camera (if all then we have an imu issue).

Hi there, just a quick sanity check. Does the drone fly normal in manual mode?

If so, it is worthwhile to do a hand/bench test by powering up on a flat surface. Then connect to the drone via wifi and startup VOXL-Portal. Going to the VIO panel in portal, if you life the drone up and down, then left and right, it should be reflected in portal with the correct trajectory display. If not and it drifts in manual mode too, then there's like a voxl2 calibration or param issue (not vio).

@berayksl Hi there. being 1.4.3, you may want check the parameter in /etc/modalai/voxl-vision-hub.conf that:
"en_vio": true,

Also, on some setups we noticed added latency that triggers the PX4's EV timeout and could explain the "invalid setpoint" log message. In PX4 please set EKF2_NOAID_TOUT parameter in QGC to 8000000.

Checking the above and before your next test flight, perform a hand test as follows:

1. startup voxl portal & connect to the drone (web browser w/ip addr of the drone as the url--wifi must be running)
2. goto the "vio" tab in portal
3. pick up the drone and move it back and forth, then left and right. You should get something close to a "plus" trajectory in the portal screen (see attached). If it "flies away" then good chance calibration (imu, or camera, extrinsics) is off. If it stays frozen/unresponsive then there's a SDK configuration error.
4. If you can repeat what is shown in the image, that confirms VIO is working properly and likely the above NOAID issue..
   
   --
   --

if you're still having issues on take off, then you can modify in /etc/modalai/voxl-open-vins-server.conf and try the blind takeoff option by setting "en_vio_always_on": false, but note VIO will not report any movements (vio position will report as if frozen) until you arm the drone.

@tkddnjs825 Hello, can you post the output from the command voxl-version. We urge all starling users to migrate to SDK 1.4.1.

@SKA Yes, VFT is actually deprecated in 1.4.1

Please replace your ovins config with this and give it a try:

@sssagara HI there, the default feature tracking settings have changed since 1.3.5 as we have found performance degradtion in challenging light conditions. I suggest 2 choices:

• in voxl-open-vins-server.conf set en_ext_feature_tracker to false (which will increase cpuload may affect your/3rd party applications, ros, etc...)
  or
• we urge upgrade to SDK 1.4.1 with its improved feature tracker configuration that will correct these take off issues (preferred)

We have noticed that sitting unarmed before takeoff, if there are no features (< 3) or features jumping around (3 <-> 12+) when one arms the motors for takeoff: that will cause ovins to drift off due to lack of features (dead reckoning). If you can run a screen record of voxl-portal's ov_overlay screen or run voxl-logger -o and post that log we can confirm that is your issue and can provide necessary steps to correct.

@star123 At this time masks are supported through programmatic means only.

I'll add it to our feature list for the next release and should be in our nightly builds by mid-next week. If you take a look at /data/modalai/zero_ref.jpg is an example format to give you an idea when the feature is ready.

@SKA Can you post your output from running on the drone:
voxl-version?
As well as what aircraft model is this? (appears to be Sentinel?)

Older versions (pre Jan 2025) we may need to have yo run the ovins internal-mode tracker. Otherwise it maybe worth to upgrade to the latest SDK.

@Kessie

Again, longer waiting times on the ground during pre-flight (<2min) will introduce drift either by thermal issues or feature noise.

To address thermal issues, the IMU is not temperature calibrated from the factory. You can do so by running
voxl-calibrate-imu-temp

To address feature noise, you can raise the drone a couple inches above the ground (typically 3 inches).

If both cases cannot be done in your application, we do offer a blind takeoff feature in ovins, by setting in voxl-open-vins-sever.conf:
"en_vio_always_on": false
But note, you will see vio stay 0,0,0 (even if you move the drone) and see any feature points freeze in place as this feature holds the vio state at 0,0,0 until the drone is armed and very slow takeoffs can introduce a static Z offset upto 0.1m.

Hi there, typically when OVINS is resetting constantly, that means there's no image stream or feature data being sent to it. Since we see a video the image stream is ok.
You want to run in a adb shell:
a. cat /etc/modalai/voxl-open-vins-server.conf and confirm en_ext_feature_tracker: true
b. run voxl-inspect-services and ensure voxl-feature-tracker is running. You can also check voxl-portal and confirm a feature_overlay video feed is present.
c. then restart voxl-open-vins-server -d as you did above.
The drone should be on a level surface when ovins is initializing.

If that doesn't work, can you post your /etc/modalai/vio_cams.conf.

@Kessie Hi there, can you please post your vio_cams.conf file?

So far is appears the Quality of the vio data is degrading in both flights, just faster in the bad flight. Is there a reason to sit in the takeoff state (on ground, motors spinning) for 20+ seconds? A scenario like that will create a level of vibration/movement in the cameras where ovins will start to pick up false positive features creating instability. We can increase the threshold for takeoff, but may give an undesirable offset of a origin (0,0,0 maybe 1/2m above the ground for example). You can first try increasing the following values by 50% increments:
"zupt_max_velocity": 0.03,
"zupt_noise_multiplier": 1,

Hence:
"zupt_max_velocity": 0.045,
"zupt_noise_multiplier": 1.5,

And see if that improves.

@Allen-Wu Yes, there are multiple ways to use GPS hardware and data all controlled by PX4. Let me walk you through the possibilities in the order of coarse grain control to fine grain control below:

1. Unplugging the connector to the GPS unit: this will outright disable its capability (also losing magnetometer as well). In this case no gps data will be present in the system nor logged. This applies from power up to power down.

2. Disabling the GPS driver, done by using the PX4Shell and running "gps stop': GPS hardware is still running but data is not used by PX4 subsystems, hence why you still get raw data logged. You can access the PX4Shell via QGC-->[Analyze Tools]-->[MAVlink Console] (type 'help' will show you the command options). This is the scenario in the above video example.

3. Enable/Disable GPS processing: apply PX4 params.
   Apply params using these parameters will include GPS data in the position estimate along with VIO.

4. Removing GPS data in flight: you can just alter these 2 values to add/remove GPS data in the position estimate.
   EKF2_EV_CTRL
   EKF2_GPS_CTRL
   I'll refer to the QGC/PX4 docs on their function. note altering these in flight can cause jumps in the position estimate causing the drone to have aggressive, unpredictable motions until the position is reestablished.

From your request, it sounds like you want both VIO and GPS running, hence you can just apply option #3 until you're ready to turn off GPS.

Hi there, if you're using GPS+mag+vio then you'll need the following PX4 params changes in addition to our vio_gps_baro.params helper file:

EKF2_MAG_TYPE = automatic (voxl default)
EKF2_MAG_CHECK = 1 [or 2/3 for more sensitivity] (voxl default is 0)

Having mag_check = 1 will activate EKF2's field strength checker such that as you proceed indoors should stop mag fusion completely when the field strength goes outside thresholds (hardcoded within PX4 ~ .4gauss variance). It's similar to GPS's DOP threshold where we expect both (gps and mag) to exceed the thresholds where PX4 will stop fusion of those modalities and eventually use vio solely.

PX4's logic tries to maintain "smooth flight", so expect some delay/drift as it starts turning on/off the above (i.e. it will not be instant). Hence, adjust performance in those transitions, you'll need to modify ekf2's baro, gps, and mag *_GATE params accordingly.

Note PX4 will adopt whatever heading is being derived from the activated sensors (GPS/Mag/vio) to construction its "global frame", so proceeding indoors you will still be operating under a global frame coordinate system (will still have a north reference based on last good mag/gps check).

Hey there, voxl-logger for openvins is only avaiable using a combination of raw flags, e.g. -c tracking_front -i imu_apps -r ov -r ov_extended -r ov_overlay Yes the -o option for odometry is not completed yet for openvins. That hsould get you the log data for openvins

As for simulating camera stream failure, it's very likely any attempt to cover a camera will increase the exposure of that stream, and in some cases you will still see random features: for example, covering with a finger will still pick up features from your finger print. We're still actively improving voxl-feature-tracker and should have a new version available soon.

@Jetson-Nano If using QGC for mission planning, indoor missions are more of an approximation since openstreetmap displays no indoor spaces (e.g. approximating waypoints on the roofline of buildings). If there's a way to overlay an indoor floorplan/raster or add custom maps using a WMS server into QGC would be ideal. Then you can ingest a floor plan of a indoor space and perform precise waypoint planning. Note QGC mission planning provides distance info between waypoints, so one can construct an fairly accurate flight pattern (e.g. for navigating known aisleway & starting point for example).

To describe what's happening under the hood: we're using QGC's takeoff mission item as the local position origin for VIO, and its registered lat-lon is used as a global position offset (though you can manually type it in as well in QGC). From there, waypoints are executed in reference to that origin and true north (of the map). In the end, you're basically flying a pattern based on the take off point and true north. Hence, it doesn't matter where you fly on the map cause this feature is telling PX4 where the drone is located in global space (when running vio only)--and executes the same flight pattern [on the map] from the distances & heading.

Alternatives for indoor "mission planning": we do offer voxl-mapper (trajectory-style) as well as wps mode in voxl-vision-hub (waypoint-style), though both are offboard mode planners in local space and thus precise by setup, but being offboard-mode based, you do not get all the safety measures that PX4 mission mode provides (e.g. RTH, PL, Geofencing, Auto-mode, etc...).