Posts made by Cliff Wong

@tkddnjs825 Hello, can you post the output from the command voxl-version. We urge all starling users to migrate to SDK 1.4.1.

@SKA Yes, VFT is actually deprecated in 1.4.1

Please replace your ovins config with this and give it a try:

@sssagara HI there, the default feature tracking settings have changed since 1.3.5 as we have found performance degradtion in challenging light conditions. I suggest 2 choices:

• in voxl-open-vins-server.conf set en_ext_feature_tracker to false (which will increase cpuload may affect your/3rd party applications, ros, etc...)
  or
• we urge upgrade to SDK 1.4.1 with its improved feature tracker configuration that will correct these take off issues (preferred)

We have noticed that sitting unarmed before takeoff, if there are no features (< 3) or features jumping around (3 <-> 12+) when one arms the motors for takeoff: that will cause ovins to drift off due to lack of features (dead reckoning). If you can run a screen record of voxl-portal's ov_overlay screen or run voxl-logger -o and post that log we can confirm that is your issue and can provide necessary steps to correct.

@star123 At this time masks are supported through programmatic means only.

I'll add it to our feature list for the next release and should be in our nightly builds by mid-next week. If you take a look at /data/modalai/zero_ref.jpg is an example format to give you an idea when the feature is ready.

@SKA Can you post your output from running on the drone:
voxl-version?
As well as what aircraft model is this? (appears to be Sentinel?)

Older versions (pre Jan 2025) we may need to have yo run the ovins internal-mode tracker. Otherwise it maybe worth to upgrade to the latest SDK.

@Kessie

Again, longer waiting times on the ground during pre-flight (<2min) will introduce drift either by thermal issues or feature noise.

To address thermal issues, the IMU is not temperature calibrated from the factory. You can do so by running
voxl-calibrate-imu-temp

To address feature noise, you can raise the drone a couple inches above the ground (typically 3 inches).

If both cases cannot be done in your application, we do offer a blind takeoff feature in ovins, by setting in voxl-open-vins-sever.conf:
"en_vio_always_on": false
But note, you will see vio stay 0,0,0 (even if you move the drone) and see any feature points freeze in place as this feature holds the vio state at 0,0,0 until the drone is armed and very slow takeoffs can introduce a static Z offset upto 0.1m.

Hi there, typically when OVINS is resetting constantly, that means there's no image stream or feature data being sent to it. Since we see a video the image stream is ok.
You want to run in a adb shell:
a. cat /etc/modalai/voxl-open-vins-server.conf and confirm en_ext_feature_tracker: true
b. run voxl-inspect-services and ensure voxl-feature-tracker is running. You can also check voxl-portal and confirm a feature_overlay video feed is present.
c. then restart voxl-open-vins-server -d as you did above.
The drone should be on a level surface when ovins is initializing.

If that doesn't work, can you post your /etc/modalai/vio_cams.conf.

@Kessie Hi there, can you please post your vio_cams.conf file?

So far is appears the Quality of the vio data is degrading in both flights, just faster in the bad flight. Is there a reason to sit in the takeoff state (on ground, motors spinning) for 20+ seconds? A scenario like that will create a level of vibration/movement in the cameras where ovins will start to pick up false positive features creating instability. We can increase the threshold for takeoff, but may give an undesirable offset of a origin (0,0,0 maybe 1/2m above the ground for example). You can first try increasing the following values by 50% increments:
"zupt_max_velocity": 0.03,
"zupt_noise_multiplier": 1,

Hence:
"zupt_max_velocity": 0.045,
"zupt_noise_multiplier": 1.5,

And see if that improves.

@Allen-Wu Yes, there are multiple ways to use GPS hardware and data all controlled by PX4. Let me walk you through the possibilities in the order of coarse grain control to fine grain control below:

1. Unplugging the connector to the GPS unit: this will outright disable its capability (also losing magnetometer as well). In this case no gps data will be present in the system nor logged. This applies from power up to power down.

2. Disabling the GPS driver, done by using the PX4Shell and running "gps stop': GPS hardware is still running but data is not used by PX4 subsystems, hence why you still get raw data logged. You can access the PX4Shell via QGC-->[Analyze Tools]-->[MAVlink Console] (type 'help' will show you the command options). This is the scenario in the above video example.

3. Enable/Disable GPS processing: apply PX4 params.
   Apply params using these parameters will include GPS data in the position estimate along with VIO.

4. Removing GPS data in flight: you can just alter these 2 values to add/remove GPS data in the position estimate.
   EKF2_EV_CTRL
   EKF2_GPS_CTRL
   I'll refer to the QGC/PX4 docs on their function. note altering these in flight can cause jumps in the position estimate causing the drone to have aggressive, unpredictable motions until the position is reestablished.

From your request, it sounds like you want both VIO and GPS running, hence you can just apply option #3 until you're ready to turn off GPS.

Hi there, if you're using GPS+mag+vio then you'll need the following PX4 params changes in addition to our vio_gps_baro.params helper file:

EKF2_MAG_TYPE = automatic (voxl default)
EKF2_MAG_CHECK = 1 [or 2/3 for more sensitivity] (voxl default is 0)

Having mag_check = 1 will activate EKF2's field strength checker such that as you proceed indoors should stop mag fusion completely when the field strength goes outside thresholds (hardcoded within PX4 ~ .4gauss variance). It's similar to GPS's DOP threshold where we expect both (gps and mag) to exceed the thresholds where PX4 will stop fusion of those modalities and eventually use vio solely.

PX4's logic tries to maintain "smooth flight", so expect some delay/drift as it starts turning on/off the above (i.e. it will not be instant). Hence, adjust performance in those transitions, you'll need to modify ekf2's baro, gps, and mag *_GATE params accordingly.

Note PX4 will adopt whatever heading is being derived from the activated sensors (GPS/Mag/vio) to construction its "global frame", so proceeding indoors you will still be operating under a global frame coordinate system (will still have a north reference based on last good mag/gps check).

Hey there, voxl-logger for openvins is only avaiable using a combination of raw flags, e.g. -c tracking_front -i imu_apps -r ov -r ov_extended -r ov_overlay Yes the -o option for odometry is not completed yet for openvins. That hsould get you the log data for openvins

As for simulating camera stream failure, it's very likely any attempt to cover a camera will increase the exposure of that stream, and in some cases you will still see random features: for example, covering with a finger will still pick up features from your finger print. We're still actively improving voxl-feature-tracker and should have a new version available soon.

@Jetson-Nano If using QGC for mission planning, indoor missions are more of an approximation since openstreetmap displays no indoor spaces (e.g. approximating waypoints on the roofline of buildings). If there's a way to overlay an indoor floorplan/raster or add custom maps using a WMS server into QGC would be ideal. Then you can ingest a floor plan of a indoor space and perform precise waypoint planning. Note QGC mission planning provides distance info between waypoints, so one can construct an fairly accurate flight pattern (e.g. for navigating known aisleway & starting point for example).

To describe what's happening under the hood: we're using QGC's takeoff mission item as the local position origin for VIO, and its registered lat-lon is used as a global position offset (though you can manually type it in as well in QGC). From there, waypoints are executed in reference to that origin and true north (of the map). In the end, you're basically flying a pattern based on the take off point and true north. Hence, it doesn't matter where you fly on the map cause this feature is telling PX4 where the drone is located in global space (when running vio only)--and executes the same flight pattern [on the map] from the distances & heading.

Alternatives for indoor "mission planning": we do offer voxl-mapper (trajectory-style) as well as wps mode in voxl-vision-hub (waypoint-style), though both are offboard mode planners in local space and thus precise by setup, but being offboard-mode based, you do not get all the safety measures that PX4 mission mode provides (e.g. RTH, PL, Geofencing, Auto-mode, etc...).

@Jetson-Nano Hi there, here's a tutorial on using this feature.

The video applies one of the vins_vio params-- based on if you are planning indoor or outdoor missions in QGC. The drone will automatically acquire heading from the internal compass. Note, the compass is not used for position estimation, just to acquire vehicle heading at power up. This is required in order to assign a global heading "at origin" (aka takeoff) since we are dealing with QGC's global Lat Lon system for mission planning. This allows a saved QGC-GPS mission be reused as long as a takeoff position is added....and loaded and executable under vio-only position estimation of course.

As an alternative, one may not use the above params, but instead use the option of manually entering in the heading of the vehicle. That is done in the takeoff item's heading field of a QGC mission. That is to allow manually heading change on the fly.

Hi there, yes, unfortunately the repo for OpenVINS is still under construction as it's under active development by the ModalAI team. We'll have it available soon

voxl-open-vins-server does a similar process to the ROS implementation when using camera data--in order to sync frames, data comes in a stereo-type packet. During camera calibration, instead of Kalibr, we are using our own voxl-calibrate-camera implementation that comes with the SDK. And as needed, one can turn off the stereo output in voxl-camera-server and calibrate each camera individually. That would be best on the ROS side as well where you can use split the images from camera-server like this before publishing to Kalibr (where 2 topics work best for openvins), then calibrate independently.

You could try voxl-calibrate-camera instead and yes it will output opencv-format intrinsics where you can copy the needed values into the ROS launch file. We create extrinsics using mechanical design. In both cases, OpenVINS will refine them at runtime during the first few seconds of operation.

You can upgrade to the latest dev version via editing your /etc/apt/sources.list.d/modalai.list on voxl2 to point to the dev repo. Then run apt update && apt upgrade voxl-open-vins-server. Looking at the /etc/modalai/voxl-open-server.conf may give you some ideas on how the roslaunch file can be setup as well.

@Jetson-Nano
Hey there, yes, if you are getting the QGC message that the "system is not ready" and will not let you arm in position mode, then you need a patch that sends the correct heartbeat message in PX4 to allow arming. It is in the SDK1.1.1 release, So I believe an apt-get upgrade should solve your problem.

But if needed the code is described here.

Hi all, couple things to clear up on this feature:

With the above parameters you are both setting limits for obstacle avoidance (OA) and collision prevention (CP)--2 different features. Details of each are here: OA and CP

To enable obstacle avoidance, make sure these parameters are being used.

Now collision prevention takes the additional step where you need to set param COM_OBS_AVOID=1. If you get a "Avoidance system not ready" message then that is a know s/w bug that is will be available on our development updates repo shortly. Also note these modes only work when MPC_POS_MODE is 0.

Lastly to disable OA, use these parameters.

Hi there, do you have COM_OBS_AVOID=1 (enabled)? You can set the parameter directly (params) or there's a menu option in QGC.

If so, can you arm the drone? With *_AVOID=1 I suspect all will work as you are seeing obstacle_distance data on mavlink.
If not, then you should get a QGC report "Avoidance system not ready"--let us know which case you end up with.

Hi there, looks like there's some confusion with what mode is being tested. Here's some clarification. PX4 has 2 approaches to handling obstacle distance data:
a. Obstacle Avoidance(OA) feature which stops the aircraft if object detection occurs from the voxl-voa service. This requires CP_* parameters set that you can compare with your drone listed here based on your application.

b. Collision Prevention(CP) feature, which not only includes OA, but also rerouting capability in all autonomous modes (position, alt, mission, offboard) and leverages the same voxl-voa service. This requires not only the CP_* parameters but also requires COM_OBS_AVOID = enable.

If your application requires (a), you can leave com_obs_avoid = disable and OA should work out of the box as is. If your application requires (b), then we have discovered a software bug in the VOA service. And the main reason you get the "collision module not ready", no arm possible. It has been corrected [code here] and binaries will be available in our gitlab daily builds shortly.

@AP123 Hey there. Are you running VOA against a stereo camera pair or via a ToF sensor?

If you run voxl-inspect-points voa_pc_out and see data, then the voxl2 SDK services are good and generating the correct mavlink obstacle distance points to PX4. If not then there's a configuration issue.

If that is good, then you need to make sure PX4 has
MPC_POS_MODE = Simple pos control or smooth pos control, COM_OBS_AVOID is enabled, CP_DIST > 0 (e.g. 2) and you must have an established home position (or good position from qvio if that is the case). Typically an "obstacle system not ready" is due to the MPC_POS_MODE (I think we default to acceleration-based input).

If all those check out, then it's likely missing a home position requirement in PX4 & not voxl, and we'll need to look that up.

Hey there, if you are flying solely with VIO (no GPS), suggest loading our indoor estimator params to add to the current setup on PX4. Outdoor vio+gps is here. These set the vio sensitivity that is in line with the suggestions mention on your PX4 forum's thread. Also we're assuming that extrinsics.conf values have not changed and are same as the factory settings.

Secondly, if you run voxl-inspect-pose vvhub_body_wrt_local on voxl2 it will apply the extrinsics and output pose that should be inline w/PX4's local position. If those values are correct, then we're looking at a PX4 tuning/config issue (we can still help).

One thing that looks weird on your PX4 flight log is Roll and Yaw rates are not keeping up w/respective setpoints (noisy). [Hence, thirdly, suggest an] IMU calibration on the PX4 side.

Lastly in /etc/modalai/voxl-vision-hub.conf there's a parameter called en_reset_vio_if_initialized_inverted. It is set to true as it's to account if vio was accidentally initialized upside down (while carrying the drone for example). You can set it to false and if there's a mismatch on VIO-PX4, you'll see it immediately.