Posts made by Cliff Wong

@DronAlan
Hi there, the configuration & extrinsics files look fine (just switch offboard mode from figure_eight back to trajectory mode. Since you confirmed figure 8 and position mode are working fine and reflect true ground truth in voxl portal, next is to dive a bit deep into the processes:

Just to confirm: are you using a loaded saved map? Orientation can be different in a loaded maps based on how it was saved. Going forward in debugging this I would create the map while flying, then test plan-to-point to ensure it is working properly.

• Before take off, if you ssh onto the drone and run voxl-vision-hub --debug_offboard and rerun the mapping test (point 1m in front of the drone hovering), and you don't need to map the entire room, just get the general area.
• In the ssh session, monitor the output from voxl-vision-hub.
• Take off and fly forward and backward in position mode, with some slight yaw motions to generate a decent map in voxl portal (recall we're generating a map in flight then plan to fly to a point, not using a old-saved map).
• Then switch into trajectory mode. In the ssh session, you should get a Received trajectory has duration message printed.
• Goto voxl-portal and plan-a-point. Then execute go-to-point. In the ssh session you should get a Received insert command. printed out on the terminal. And it will show the drone's forward commands, i..e set points, to your planned point. I expect the commanding: XYZ values to increase in the positive X direction. Please post the output of the ssh session here.

If the drone moves in the backwards direction as you've been seeing but the commanding XYZs are moving forward, we have found our problem (PX4 issue such that I'll need your params file). If the drone moves forward with commanding XYZs are moving forward then it's how your older maps are being saved and we can go from there.

Hi there, just to confirm: does flying manually in position mode work and follows the XYZ position in voxl-portal's VIO page. as well as follows figure 8 mode too? This will confirm the vio pose estimate is correct and more of a issue on mapper.

@DronAlan With 1.6.3 SDK upgrade, did you test against the figure 8/offboard mode? Also flying in position mode after drawing the trajectory, if you manually fly the path you created, does the drone follow correctly? If position mode manual flying is working fine and figure 8/offboard mode is working fine (a clockwise figure then we have a trajectory calculation problem on the voxl-mapper side (I'll check our code, but our QA currently show that was working properly).

Since you're using OpenVINS vio module now, there's an unlikely chance of a reset causing a 180 orientation change by design. Also make sure qvio is disabled. You can check if it's running via voxl-inspect-services and run systemctl disable voxl-qvio-server (+reboot).

@DronAlan Hi there, when the camera was rotated 180, you'll have stable take off behavior, but any XY movement would cause the drone to 'fly away' quickly.

Taking a step back with your concerns on physical mounts:

1. checking camera orientation:
   a. bring up the front camera in portal: it should be upright orientation, e.g. top of the camera image should be from the top of the drone itself ("tops are aligned").

2. Since you are using qvio, only 1 camera is supported. Ensure in /etc/modalai/vio_cams.conf has tracking_down disabled, It's a safeguard if qvio is accidentally using that camera (would would lead to a orientation problem).

As for IMU, the voxl2 unit should have the USB-C port facing the bottom/ground side (and WiFi dongle on the top side). Let's force reset the imu calibration. If you run on the drone voxl-calibrate-imu and follow the directions, will reset the imu.

Let's check these items. If they all pass and still presents the orientation problem, then it's worth moving to the SDK1.6.3 to take advantage of the advanced config installer.

@DronAlan Hi there,

Seeing the opposite direction that looks like a 100% 180 rotation is typically a sign of a reset while powering up: incorrectly detects the gravity vector. This is a common anomaly with qvio. I suggest at power up, immediately put the drone on the ground and try not to move the drone until takeoff. Try this and see if you get consistency. QVIO is sensitive to resets prior takeoff. You can edit the following in /etc/modalai/voxl-vision-hub.conf:

        "en_reset_vio_if_initialized_inverted": false,

if your use case doesn't allow this type of procedure, you can ssh onto the drone and force reset before every takeoff by running: systemctl restart voxl-qvio-server.

If you have not edited the extrinsic settings from factory then I would safely assume they are correct, but in case, extrinsics are stored in /etc/modalai/extrinsics.conf where you'd look for the body to imu parameter section (which should be 0,0,0 in starling's case).

Note: these types of anomalies have been corrected in SDK 1.6.x, as the latest SDK now defaults to the new & improved openvins VIO subsystem if you're inclined to upgrade (recommended, though any custom-external apps make need to be recompiled as the MPA API has changed).

@DronAlan Hi there,

Can you power up, login and at the ModalAI splash screen is a label: SKU: [some text value] Please post that sku text here to help us diagnosis.

This looks an extrinsics error posting improper orientations of the drone. Please try the following, I expect one of these actions will fail:
a. power cycle and perform a bench test to ensure the extrinsics are proper (forward is X, etc..). If that passes...
b. edit file on drone /etc/modalai/voxl-vision-hub.conf and change back to figure_eight mode. Flight test in figure 8 mode (it should fly a controlled figure 8). If these tests pass, your extrinsics are good and you can switch back to trajectory mode.

If those 2 actions pass, then this leans towards a classic VIO reset error. Rerun your mapping test case, login and run voxl-inspect-qvio -n and it will provide the status if vio reset in flight. Typically the lighting conditions of one's environment are poor for qvio to work robustly and you can get multiple rests in flight.

@greg_s That would explain a lot what's happening.

If you start fresh and connect to the drone with voxl-portal and goto the VIO tab/menu option, you should be able to pick up the drone and perform a bench test.

Considering Figure 8 is not working, you will need to inspect the front and downward camera, tighten/remount any loose parts and possibly check camera focus & calibration.
Worse case if the forward camera is good and not changed from factory, you can edit /etc/modalai/vio_cam.conf on the drone and disabled the faulty camera (disabled the down camera).
You'll have degraded precision (mainly bad vio in fast yaw motions), but should be able to perform a figure 8 and should get you running until you repair that bottom sensor.

@greg_s HI there,

The video exhibits classic VIO lost behavior from extrinsics: hovers fine, then on translation just flys away. Can you switch from trajectory mode back to figure_eight mode and confirm the drone can fly in figure 8s for more than 30secs?

Figure 8 also tests the trajectory engine that mapper uses so it will confirm a number of things on the sensor side.

@greg_s said in Starling 2 not following navigation path:

One thing that looks suspect is the voxl-mapper pipe size 64k bytes is much smaller than expect (usually in MB, e.g. 32MB-64MB). Is this a preloaded 3D map?

Otherwise your VIO data looks good. It appears the drone does go to the 1st setpoint (1st leg of the trajectory) where that setpoint also looks as if it is landing. It's possible the landing detector was triggered and why everything stops.

I suggest the following:

• power up, ssh into 2 terminals

• terminal 1 -- run the voxl-mapper
  terminal 2 -- run voxl-vision-hub --debug_offboard

• take off in position mode, bring up the portal into mapper 3D view as normal, perform a 360 yaw to get a good map

• Select "Plan a Point": draw your point and position it correctly

• switch to offboard mode (which you have set correctly to 'trajectory')

• Select GO!

• Select "Follow Path"
  (If no follow path is offered then the A* path creation failed)

Hopefully with that sequence, you'll see the drone move along the path. If it does not, then the voxl-mapper output in terminal 1 will tell us if a latency is the culprit.

As an example, here's how I expect the above to play out.. If you cannot repeat this then it maybe worth upgrading to SDK 1.6.3 (as the example was generated in).

@taiwohazeez HI There,

What SDK version are you using. Anything over version 1.5.3, we recommend using the openvins VIO implementation rather than Qvio.

Typically a different XYZ value from actual indicates VIO reset during flight. What you want to try 1st is a bench test to confirm vio is set up correctly. Essentially power up the drone on a level surface, then connect to it and bring up voxl-portal (http://<ip of the drone>) and goto the VIO page. The page will show the drone in local space (0,0,0 xyz) if you pick up the drone and move it in a 3D plus pattern the vio page in portal should reflect that. If it does not, then we a configuration issue.

@greg_s
Hi there, your extrinsics are good: same as a factory Starling. Now, if you altered the voxl-camera-server.conf (e.g. enabled a flip of a camera stream when it wasn't) or physically changed a camera mount point then the extrinsics would need to be adjusted. Let assume that's not the case.

From your vvhub trajectory output, it appears your setpoints command the vehicle to move forward (X) and to the right (Y) > 1m while dropping altitude (Z) 0.8m --recall mapper runs in FRD (aka body) space where Z up is negative. Also you do pass through the take off origin (0,0,0). If that's not what you wanted, then we have a "requested trajectory" drawing problem in the portal tool or mapper's found invalid regions/obstacle and could not draw the "requested trajectory".

In the former case @ApoorvThapliyal mentions the un-intuitiveness of the portal tool that you already understand.

In the latter case, upon "request trajectory" submission, voxl-mapper will analyze the map to see if there are obstacles and alter the trajectory to fly around potential obstacles. You can alter in voxl-mapper.conf:

• rrt_min_distance (try 0.125)
• esdf_inner_sphere_radius (try 0.15)
  if you seeing mapper create trajectories going around 'ghost' objects. This should allow RRT generated trajectories to get closer to objects.

In checking precision, you'll need to take the trajectory output (e.g. the setpoints in the *commanding: XYZ* messages) and compared it against VIO position to determine tracking error. You have 2 ways to get that data:

• option 1 is record you vvhub traj output as above and record the output of voxl-inspect-vins -n then plot compare the tracking error.

• option 2 is running voxl-vision-hub --debug_offboard --debug_odometry which you'll need to seperate the traj messages from the vio (T_body_wrt_local) messages, then you ca plot the tracking error.

The goal is we want the vio position to follow the commanded trajectory: (it's doing what is told to do). That will narrow down where your concerns are w.r.t what module has a problem.

Hopefully that gets you on the right track, just post any outputs you gather that continue to show tracking error.

Hi there, I would highly recommend upgrading to the latest SDK 1.6.2.

Q1: For voxl-vision-hub, openvins is just another [abstracted] VIO client, using the same MPA interface as QVIO. Thus, should work the same as QVIO.
Q2: There are no outstanding buffer conflicts between the 2 inputs in that SDK version. Though best to update to 1.6.2 as there have been numerous MPA improvements.

Q3: MAYBE ON GROUND, POSITION LOCK? is reporting that the barometer is reporting <1m and OVINS > 1m (-1 in NED space). It appears OVINS had a blow up, has recovered with a new origin in the local frame. Your output says it must have reseted in the ai/flying and hence the new local origin is above your landing point (fixed frame). Depending how you have PX4's EKF setup could lead to issues like the landing detector not working, or position error when running missions, hence why it's reported as a warning. Your output says it's at 0,0,-15 indicating OVINS restarted at ~15m up, but the PX4 baro says it's at 0. If outdoors, I recommend setting EKF2_HGT_REF = baro, if indoors, you'll need to reconcile a origin offset if VIO resets midair (qvio would require this as well) if using vio data directly waypoint manager (i.e. offboard waypoint mode or custom nav code)

Going forward if staying with 1.4.x,

1. edit /etc/modalai/voxl-open-vins-server.conf and change:
   "en_vio_always_on": true,
2. ssh into the drone and run voxl-vision-hub --debug_fixed_frame --debug_tag_cam --debug_odometry to compare values and see if there is a error in calculation possibly causing vision-hub to restart.
3. move the drone around by hand, point it at a tag as needed and monitor the output from vision-hub. Should tell you where a possible conflict arises.

option 2: upgrade to SDK 1.6 as we have a new version of VIO and numerous bug fixes

Directions are found here. where the SDK for starling 2 is found here.

Hi there, looking at the files and review our analytics here since you were flying fine in position mode that says VIO was likely ok (as per you config files).

We did have a some changes between vision-hub and px4 that make have been incompatible with the 1.4.5 version. Hence you have 2 options:

option 1: debug & confirm either probem lies in voxl-vision-hub or PX4.
(a) you need to ssh onto the vehicle, in the remote terminal, run:
voxl-vision-hub --debug_offboard

That will display streaming position data, which is the traj commands to PX4. Then you may restart/start voxl-mapper (e.g. systemctl restart voxl-mapper). In the vision-hub streaming output, you'll see a print out like Trajectory Monitor connected to voa pointcloud.

(b) you can now switch to position mode, fly around in that mode to map as normal, then stop to hover. Draw your simple trajectory in portal and execute. Ensure the draw trajectory is within the space using the mouse and rotating around the area--portal can sometimes make the 'plan point' appear close to the drone, but it is actually very far away so you need to check at different viewing angles. When you execute the plan, the streaming output in the vision-hub display above should reflect the drawn trajectory. In both your example and mine, the Z position output should stay relatively close to the current. If it rises, we have a trajectory calculation problem (drone is doing what it's told). If it rises while the position data is fairly minimal, then PX4 likely has failed (VIO or dropped out of offboard mode). Please try this and report back. We suspect VIO may have failed. Here's a example of the output:

@greg_s Hi there, can you post these 2 files from your drone:

/etc/modalai/voxl-vision-hub.conf
/etc/modalai/voxl-open-vins-server.conf

One thing from your screenshot is I sort of see the drone's position and the trajectory planned. For example as in this screenshot, the drone's xyz position should be visible (xyz axes) and a trajectory (black & red lines) should be displayed before executing a trajectory. The drone should be moving in realtime hovering in position. If it's static the vio was likely lost.

@berayksl Hi there, to address your questions:

• The inconsistent VIO performance between flights

appears the inconsistent vio behavior on multiple arm/disarms is due to the way voxl-open-vins-server re-initializes after a disarm event. Are you still using in the voxl-open-vins-server.conf option "en_vio_always_on": false and "en_auto_reset": true?

If so, the next thing that can be an issue is takeoff speed. To ensure VIO has the correct origin (0,0,0) we have a conf parameter called: "takeoff_accel_threshold": 0.5, This is essential a takeoff detector and is set for a aggressive take off (>80% throttle)

If you are taking off very slow and gentle, it's possible the detector is not being tripped. Try lowering it from 0.5 to 0.3.

• The “Yaw estimate error” message (should I be concerned if QGC shows no error)?

If the error doesn't repeat, it's just VIO telling PX4 to reset it's heading and PX4 complains on reset. This can be ignored. BUT if QGC keeps repeating the announcement, you have a PX4 yaw_gate issue--make sure in PX4 the param EKF2_EV_CTRL has the 'yaw' option checked.

And as a sanity check cause you are seeing an R/C issue: goto the radio tab in QGC and confirm your pitch/roll/yaw inputs are calibrated and centered such that you are not sending biased commands.

• The drone ignoring RC inputs during the second flight

Are you getting a r/c "manual control lost" announcement in QGC or just lack of control when trying to move the drone? On the former case we have notice users in environments with lots of metal (e.g. warehouse) will have RF interference issues.

At this point, we have numerous improvements in SDK 1.5.0 for Max and it maybe worth the upgrade if your system design allows it. It has improvements with the vio-always-on issues, vio resets, and some PX4 altitude improvements.

@berayksl Hi there,

In the VOXL2 file /etc/modalai/vio_cams.conf all the cameras should be listed as a JSON record. the top-level disable attribute of each record can be edited to turn cameras on and off (make front true and the others false for just the front camera).

On calibration, instructions can be found here.

If the camera's physical positions changed from delivered, then you'll need to update the extrinsics.

@berayksl
Hi there,

1. Since you were able to run a limited bench test, having it fly away after a couple movements indicates 3 possibilities:

• extrinsics are incorrect -- either bad imu_apps to cam# values or camera outputs are flipped.
• intrinsics are incorrect (bad camera calibrations)

2. yes you can run without mag enabled. Should work normally if (of course) VIO is running normally.

3. As for PX4 parameters, VIO enabling can be setup applying this param file.

My thinking is the camera extrinsics maybe off or mismatched to the camera ID. If you can only enabling one camera at a time and run the bench/hand test as if like a single camera vio setup, hopefully it will reveal the misconfigured camera (if all then we have an imu issue).

Hi there, just a quick sanity check. Does the drone fly normal in manual mode?

If so, it is worthwhile to do a hand/bench test by powering up on a flat surface. Then connect to the drone via wifi and startup VOXL-Portal. Going to the VIO panel in portal, if you life the drone up and down, then left and right, it should be reflected in portal with the correct trajectory display. If not and it drifts in manual mode too, then there's like a voxl2 calibration or param issue (not vio).

@berayksl Hi there. being 1.4.3, you may want check the parameter in /etc/modalai/voxl-vision-hub.conf that:
"en_vio": true,

Also, on some setups we noticed added latency that triggers the PX4's EV timeout and could explain the "invalid setpoint" log message. In PX4 please set EKF2_NOAID_TOUT parameter in QGC to 8000000.

Checking the above and before your next test flight, perform a hand test as follows:

1. startup voxl portal & connect to the drone (web browser w/ip addr of the drone as the url--wifi must be running)
2. goto the "vio" tab in portal
3. pick up the drone and move it back and forth, then left and right. You should get something close to a "plus" trajectory in the portal screen (see attached). If it "flies away" then good chance calibration (imu, or camera, extrinsics) is off. If it stays frozen/unresponsive then there's a SDK configuration error.
4. If you can repeat what is shown in the image, that confirms VIO is working properly and likely the above NOAID issue..
   
   --
   --

if you're still having issues on take off, then you can modify in /etc/modalai/voxl-open-vins-server.conf and try the blind takeoff option by setting "en_vio_always_on": false, but note VIO will not report any movements (vio position will report as if frozen) until you arm the drone.