Posts made by zauberflote1

@dvz Hi would you be able to confirm which airframe and SDK versions you are using?

@Muqing-Cao

You should be able to download it and install it, as our betas are now public.

@Muqing-Cao said in starling 2 max qvio vs open-vins performance issues:

do we have to change the configs file to the startling-specific configs after update?

Not necessarily, unless you changed the default values. The new SDK installation will prompt you to identify your platform.

@Muqing-Cao
Please update it to the latest version, 1.6.3.

PS: 1.6.3 is still in Beta phase, but it should be released soon

@Muqing-Cao

Can you confirm your SDK version?

@YUUJI-INOUE
Glad to hear!

All the best,
ZBFT

@YUUJI-INOUE

Happy New Year!
I wanted to take some time to review the video and the configs you’ve posted. Here’s my personal interpretation based on the information provided.

• It doesn’t seem you’re using VFC-position, but PX4 Position mode. Recall that on Stinger the recommended default is VFC-Position. (You can change en_vio to false in VVHUB config and adjust your transmitter settings to reproduce the default configuration.)

• Your position lock break is likely due to SLAM features being rejected during the MSCKF->SLAM promotion and/or not being “rematched” properly. This strongly suggests calibration issues, particularly CAM-IMU extrinsics. Did you use Kalibr? If so, would you be able to post your Kalibr-generated report? Still, you can likely mitigate this at the expense of VINS' accuracy by bumping the chi2 value for SLAM features.

Also, the number of features you see in the inspect-vins tool represents valid SLAM features, so it doesn’t mean that the estimator has no features at all, as MSCKF features can still be available.

@jakkkkobo

Hello,
We currently do not ship any mask files with our airframes, as they are normally not required. In addition, you should be able to use a mask with minor modifications on the server side, namely VoxlCam directory inside voxl-open-vins-server repo. (The mono camera case should be a good starting point, as we already have the masking logic for occluded cams during takeoff.)

Furthermore, as we use a custom interface/backend for OV, loading the mask as done in the traditional OV-ROS format won’t work as intended, i.e., via Yaml.

All the best,
ZBFT

@YUUJI-INOUE
Hello there,
Here are a few things to be aware of:

• VVHUB "Error" for missing extrinsics for imu_body

This is not really an error, just tells you that the IMU in body frame and the body frame itself are connected by the identity. In other words, there's no transformation needed for the IMU data from VVHUB standpoint.

• Position Mode not engaging

Assuming you're able to get the drone arming in Manual/Altitude mode. Are you using VFC in your VVHUB conf or using the classic PX4 position mode? -- If your drone resembles the Stinger D0013, VFC is the suggested format. -- you can use the wizard to configure VVHUB accordingly. Can you confirm your VVHUB setup? Please attach your vision-hub and vfc .conf

• VIO not working

As you described, different frames, different noise characterization. You can change the IMU noise values inside the kalibr_imu_chain.yaml (in the directory you found estimator.yaml). Personally based on your description, I am inclined to believe your performance issue is rooted in your extrinsics values and VVHUB configuration. Did you use Kalibr? -- If you used Kalibr with imu_body pipe, all you have to do is to replace the SE(3) elements inside kalibr_imucam_chain.yaml and make sure the sync_config flag inside voxl-open-vins-server.conf is set to false. Otherwise, unless you have a proper CAD model, physically measured values will likely be inaccurate and cause the estimator to diverge. For future help, please attach all your VIO-related yamls and your voxl-open-vins-server.conf

All the best,
ZBFT

@YUUJI-INOUE
Please update your SDK version.

Regarding your rotation concern, on D0013 the board the is mounted upside down, but the common frame assumes the board is mounted in the default orientation. For instance, consider the rotation listed in the following file for imu_apps and body

/etc/modalai/extrinsics.conf

If you transform IMU samples by the SE(3) trafo, you'll see they will align with FLU coordinate frame. A quick sanity test would be to check a static IMU sample with inspect-imu and rotate it by the above-mentioned trafo. I should also add that on our soon-to-be-released SDK 1.6.3, we included some flavours of IMU pipes, including body frame, etc. You should be able to access this here, in case you wish to build it on your own.

In regards to VIO, the latest OV-server is significantly better than previous versions. Additionally, if you load the latest imu-server and OV server, you should get all estimates in body frame wrt. {G}. PS: All extrinsics are corrected internally, so you don't need to manually edit the .conf files.

@Muqing-Cao
Can you confirm your SDK version?

@remill
You can also add a FetchContent CMake declaration that will build your library during compilation. Here is an example for yaml-cpp

include(FetchContent)

# Fetch yaml-cpp from GitHub
FetchContent_Declare(
  yaml-cpp
  GIT_REPOSITORY https://github.com/jbeder/yaml-cpp.git
  GIT_TAG yaml-cpp-0.6.3
)

# Disable yaml-cpp tests
set(YAML_CPP_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(YAML_BUILD_SHARED_LIBS ON CACHE BOOL "" FORCE) #cross 4.0 quits the chat if building static
FetchContent_MakeAvailable(yaml-cpp)
# Confirm target exists
if (TARGET yaml-cpp)
    add_library(yaml-cpp::yaml-cpp ALIAS yaml-cpp)
endif()

install(FILES
    ${yaml-cpp_BINARY_DIR}/libyaml-cpp.so
    ${yaml-cpp_BINARY_DIR}/libyaml-cpp.so.0.6
    ${yaml-cpp_BINARY_DIR}/libyaml-cpp.so.0.6.3
    DESTINATION ${CMAKE_INSTALL_LIBDIR}
)

To clarify, our current platforms are shipped with voxl-open-vins-server as the default VINS solution; however, QVIO is still available (SDK 1.6.2).

@Rowan-Dempster Hi there,
Glad to hear you are exploring the new voxl-open-vins-server.

A few things to consider, as you are not using a "ModalAI airframe".

• OpenVINS follows the classic Kalibr calibration model (both cameras and IMU)
• Calibration involves IMU intrinsics, Cam Intrinsics, IMU-CAM extrinsics

From this point, I'll assume you're using a VOXL2 Board. Hence, your IMU intrinsics should be "1:1" to the values reported in here. However, depending on your airframe vibration you may need to tune the noise density and random walk values -- Personally, I would leave this as a final option, as these are a result of extensive calibration routines and have shown to be invariant among our airframes.

Moreover, you can obtain the camera intrinsics using our voxl-camera-calibration repo. I highly recommend to re-calibrate your cameras, making sure your reprojection error is in the subpixel range.

Next, we need the extrinsics from each CAM to IMU. Ideally, you may want to run Kalibr or you can use the sync_config flag in /etc/modalai/voxl-open-vins-server.conf as true under the assumption your extrinsics.conf is setup correctly. If you opt to use Kalibr, you will need to update the camchain file and make sure your sync_config flag is set to false.

Finally, we need to address the most important config file. Tuning this file may be a bit challenging because it revolves around use-case that being said, I would leave as is for the most part, addressing ZUPT params and chi2_multipliers/sigma_px based on simple hand-held tests. Particularly, if the estimator blows up when the drone is sitting still right after a power-cycle -> ZUPT params, and/or if due to heavy dynamic motion MSCKF features are losing track frequently or the platform is drifting slightly -> Bump chi2_multiplier + increase/decrease (case-by-case) sigma_px (SLAM/MSCKF).

To conclude, I would recommend deactivating the thermal calibration of the IMU for initial testing, as depending on the SDK version and platform, it could increase the perceived noise in the accel. Also, I would attempt 2Cam flights using ION pipes (vio_cams.conf) before jumping to 3Cam, as the 3rd Cam seems to add more noise into estimator if placed in opposite direction to another Cam (e.g., front and rear) and the extrinsics are not "perfectly" tuned.

PS: We have a converter from voxl-logs to rosbags for seamless integration with Kalibr, feel free to check it out

PS2: Regarding camera time-synchronization, if your cameras are being driven by an fsync line, you shouldn't have to worry about the sync dt; however, if they are not, you may want to bump the fusion_rate_dt_ms flag (voxl-open-vins-server.conf) to a value that guarantees that every camera has one individual frame. Moreover, if the timestamps are not monotonically increasing or the cameras don't start exposing at the same time -- implying that the 3Cams have non-aligned timestamps, you may need to "hot-fix" your timestamps to get a stable performance. As for the CAM-IMU dt, we use the online calibration inside of our OpenVINS implementation to estimates the value.

PS3: We currently don't have any external-release report for performance comparison with QVIO, but internally we have seen significant differences in behavior between the two (especially in multi-camera setups and under aggressive motion). In general, OpenVINS gives you more tuning flexibility and multi-camera support, at the cost of being more sensitive to calibration quality.

@Daniel-Rincon,
Please consider updating to the latest SDK release, 1.5.0, if you haven't already. This has our latest VIO public development.

As Tom mentioned, OV in 1.5.0 can use a binocular+ tracking, i.e., number of cams => 2 without stereo matching. This improves VIO robustness significantly compared to the single cam QVIO. Hence, OV should be preferred.

Additionally, I would recommend redoing a full calibration of all sensors (IMU, PX4, Cams) and checking if the extrinsics file is correct.

This should improve your flight performance.

I hope this clarifies some of your questions

Hello,
Please refer to the following and see if the problem persists:
https://forum.modalai.com/topic/4475/voxl-open-vins-server?_=1749222785639

Thanks,

@Jetson-Nano
Could you confirm that you are using the correct camera stream?

Please run the following:

voxl-inspect-cams
 voxl-inspect-vins #with openvins running on a separate terminal

Check if you have any error codes or camera feeds that have stalled.

Finally, double check your vio_cams.conf file to see if you are using the *misp_norm
camera pipes.
-- An easy way to do this is to reset your openvins and vio cam configuration files, run the following after backing up your .conf files.

 voxl-configure-cameras 
voxl-configure-open-vins

Then, reboot the platform and attempt to run openvins with the new conf files.

Hello,
Regarding the IMU temperature calibration, it is now optional as some platforms may underperform if the calibration is not executed properly, i.e., getting poor measurements at the cold phase. I would recommend removing the IMU temperature calibration file from /data/modalai and double-checking if the behaviour persists.
Quick question: Which cameras are you using? Did you set your extrinsics file correctly?
Please make sure you calibrate all the cameras as well.

Thanks,
ZBFT

Hello @Jetson-Nano,

Would you be able to confirm your SDK version?

Also, have you attempted to recalibrate the IMU?

voxl-calibrate-imu
``
https://docs.modalai.com/calibrate-imu/

All the best,
ZBFT

@Marco-Spangaro
From your test case description, there could be a few reasons for an imperfect re-initialization. Please note that monocular VINS has four unobservable directions (position and yaw), so your hand motion could likely affect initialization, especially if you are moving too erratically or performing a degenerate motion that would affect the calibration refinement.

Note that if the image frames post-reset are not feature-friendly, they could also have a negative impact.

Regarding the vio_always_on flag, some segments of the voxl-open-vins-server code will only be executed if the drone is armed or the flag is set to true.

Finally, I noticed that you increased the tracking and publishing frequency. Did you see an increase in robustness from these new parameters? Usually, for general environments, I would suggest leaving these as the default values, as you will likely have a better feature triangulation between keyframes. Additionally, enabling feature refinement can also help.

Hello Marco,
Thanks for your question.

A few things to consider:
What is your camera setup -- or is it a ModalAI development drone?

Note that the default configuration of voxl-open-vins-server does not use dynamic initialization, meaning that if a reset triggers while the platform has some motion above the maximum threshold, OV won't be able to properly re-initialize.
If you want to test if that is the case, set "init_dyn_use" to true in the voxl-open-vins-server.conf

To further diagnose the issue, please do the following:

• Run voxl-open-vins-server from terminal

voxl-open-vins-server

• Check why the reset was triggered based on the output
• Analyze the following messages post unsuccessful re-initialization -- usually, motion issues will be characterized by IMU threshold messages
• If the problem persists, upload your voxl-open-vins-server.conf file