ModalAI Forum

@Sem-Andeweg

I don't work for ModalAI but can try to help.

You need to adjust the IMU0, and IMU1 to body RPY at minimum, which stands for Roll, Pitch, Yaw.

If you look at the Seeker example at https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/master/misc_files/usr/share/modalai/extrinsic_configs/seeker_v1_voxlcam.conf#L64

you'll see the VOXL is rotated 90 degrees "up" and yawed 90 degrees right. From there the tracking camera is then rotated "down" 45 degrees to account for the VOXL CAM being mounted on the front of the Seeker and its orientation relative to the VOXL inside the camera housing.

Hope this helps a bit, definitely watch the video at https://docs.modalai.com/configure-extrinsics/

Is this a unit direct from ModalAI? If it is, it should be tuned or you could load default parameters from https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/tree/master/platforms for your version of PX4 and platform.

Then from there you load a helper through QGroundControl parameters page: https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/tree/master/helpers

That should get you close to a decent tune.

If this is a custom drone, you need to configure your extrinsics, calibrate your camera, and validate QVIO first before attempting to fly which it looks like you are attempting to do. The VOXL portal qVIO overlay will show you points selected as well as distance traveled which you can validate against manually moving the drone around prior to any attempts to fly. The camera needs to have areas of contrasting features in view -- so reflective floors or some repetitive patterns can be a challenge for qVIO.

Configure Extrinsics

ModalAI technical documentation for VOXL and VOXL 2 Companion Computers for PX4 and ArduPilot Obstacle Avoidance and GPS-denied navigation, assembled in the USA

ModalAI Technical Docs (docs.modalai.com)

Calibrate Cameras

ModalAI technical documentation for VOXL and VOXL 2 Companion Computers for PX4 and ArduPilot Obstacle Avoidance and GPS-denied navigation, assembled in the USA

ModalAI Technical Docs (docs.modalai.com)

In case anyone else comes on this thread --

https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe/-/blob/master/library/include/modal_pipe_interfaces.h#L848 defines the VIO messages within MPA.

float T_imu_wrt_vio[3]; contains the translation in x, y, z and is what is printed in the overlay within VOXL Portal.

• Positive X = Forward
• Positive Y = Left
• Positive Z = Increase in Altitude

If you then look in the Odometry messages used by PX4 or run voxl-inspect-pose these coordinates will be in FRD/NED frame of reference which is:
Positive X = Forward
Positive Y = Right
Positive Z = Decrease Altitude
(I think I've got this right -- hopefully someone from Modal can confirm)

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-vision-px4/-/blob/master/src/vio_manager.c#L350-393 walks through the publication of the odometry MAVLINK message to PX4 and rotation into appropriate frame for processing.

========
TLDR; Most of my issues for this platform were solved by disabling the QVIO server, keeping voxl-vision-px4 running, and dialing in PID for PX4 without any external inputs from vision or GPS. Once tuned, you can focus on calibrating cameras and etc/modalai/extrinsics.conf to ensure consistency in VIO inputs.

Shoutout to @wilkinsaf for tips and tricks on tuning.

Files · master · voxl / VOXL SDK / Services / voxl-portal · GitLab

Web interface hosted on VOXL

GitLab (gitlab.com)

Is an example of allocating a buffer and copying the frame into it. There are some other good examples in that file as well for processing frames and converting them to various formats within the callback. Hope that helps!

Extrinsics and QVIO server configuration will determine camera orientation and which IMU to use respectively.

Configure Extrinsics

ModalAI technical documentation for VOXL and VOXL 2 Companion Computers for PX4 and ArduPilot Obstacle Avoidance and GPS-denied navigation, assembled in the USA

ModalAI Technical Docs (docs.modalai.com)

server/config_file.h · master · voxl / VOXL SDK / Services / voxl-qvio-server · GitLab

GitLab.com

GitLab (gitlab.com)

You will need to measure from the IMU (Parent) to child (camera) for translation in meters and also the roll, pitch, and yaw in degrees. I recommend watching the video on the configure extrinsics page.

Page not found

ModalAI technical documentation for VOXL and VOXL 2 Companion Computers for PX4 and ArduPilot Obstacle Avoidance and GPS-denied navigation, assembled in the USA

ModalAI Technical Docs (docs.modalai.com)

May help you with setting up ROS on VOXL2.

https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-to-ros may also help you bridge the Modal Pipe Architecture into ROS.

Late to the party on this -- but I have been able to build ipk's off of an M1 recently.

I followed: https://docs.modalai.com/install-voxl-docker/

And then also downloaded the image from the developer portal as laid out: https://docs.modalai.com/voxl-cross/

To load:
docker load --input voxl-cross_V1.8.tar

To run:
voxl-docker -i voxl-cross:V1.8

Then the docs in all of the Modal repo's work well.

I do have buildkit etc enabled as laid out here: https://medium.com/nerd-for-tech/developing-on-apple-m1-silicon-with-virtual-environments-4f5f0765fd2f but you shouldn't need any of that. Let me know if that helps you. Worked just fine for me -- no special rosetta2 terminal or Docker version.

I'll see if I can contribute something to the docs when I have a moment.

https://forum.modalai.com/topic/1328/camera-servers-docker-build-royale-331-spectre-4-7 may have some answers for you on voxl-camera-server which is something ModalAI is working on WRT the build.