I/O Python Interfacing

PawelJ

Hello,

I have been going through the documentation for some more details on how to develop your own controllers and path planners in python, but there are quite a few sections and they mainly focus on the build process. What would be the recommended way to run python scripts on the voxl-m500? I would mainly be interested in getting state and image feedback, and sending in target information. For the controller, I would like to keep what is already running, as it works great, but would like to add an adaptive controller ontop of that that can be added to the signal being sent to the motors (whether it's speed control, pwm, etc). Lastly, will these scripts be run from a computer ssh'd into the drone, or is this something that is onboard and will be triggered when in a certain flight mode? Any guidance or example scripts are greatly appreciated.

Cheers,

Pawel J

modaltb

Hi @PawelJ ,

If you're comfortable with Docker, here's an approach most folks seem to be using in some form to get the 'base system' going:

Use an Ubuntu/Alpine Docker container on the VOXL-M500 (the M500 has the VOXL companion computer, this runs a Yocto linux, which is not as easy as Ubuntu for getting packages, so getting a container on there is an easy way to get your environment setup. Here's some docs: https://docs.modalai.com/docker-on-voxl/
The Docker container can have whatever Python you want in there. Here's an example of how we have a Python MAVSDK container: https://gitlab.com/voxl-public/voxl-docker-images/voxl-docker-mavsdk-python
VOXL is running Linux, so you can make things run on bootup using systemd etc. Here's how: https://docs.modalai.com/docker-on-voxl/#configuring-a-program-to-run-automatically-inside-a-docker-on-boot

You could also ssh over and run the container manually, or even run a web server in the container and expose an API

PawelJ

Hi @modaltb ,

Thanks for the breakdown this is very helpful. I'll have to go through those documents The web server is a very intriguing idea. Do you have any idea what kind of communication frequency you can get with that? Also, can you expand on what you mean by base system? Do you have a python api that can be used to access the modalAI functions for VIO, control, obstacle avoidance etc?

Thanks again, I have really appreciated the quick reponses from the dev team.

modaltb

We don't have a Python interface for much yet... basically an I2C interface for now, but it's slowly being added.

OK jumping around a little bit, but we do a lot in C, and a lot through named pipes most recently. New docs are coming online in the next couple weeks, but the source code is up on https://gitlab.com/voxl-public

This is a pretty clean way for getting sensor data (VIO, IMUs, etc.), via a named pipe (basically you could probably do a simple Python 'read' on a named pipe and consume use the data for what you need.

Controlling of the Flight Controller is a bit different, it's 'outside' of VOXL basically on it's own microcontroller, and interfaced via MAVLink, and handled by PX4. So you're bound to that interface (MAVLink) and features (PX4).

Not sure the rates on webserver, MAVLink to a Ground Station is likely easier when I think about it.... but there's a lot of options depending on the dev stack you're comfortable with.

PawelJ

Thank you for the abundance of helpful information I'll have to parse through all of this, but the road map helps.

Cheers,

Pawel

PawelJ

Hi @modaltb ,

I've been going through the documentation and testing various methods based on the steps you laid out above. Right now I am just focussing on setting my own path for offboard mode, and recording the VIO local_position_ned outputs to file while it is flying that path.

I was looking at the mavsdk-python package at first, but If I'm following the code in the repo correctly, it looks like it will be easier to modify and recompile the voxl-vision-px4 package. It also looks like this option will have faster communication (uart over udp?). I was hoping you could let me know if the plan of action makes sense, or if there is a simpler / more efficient way to do so.

Custom Path: Referencing the voxl-vision-px4 repo, It looks like I can add my own path generation script, similar to offboard_figure_eight.c and add it as an option in offboard_mode.c. Then I can just update the offboard mode in /etc/modalai/voxl-vision-px4.conf to my custom offboard path. The downside to this is I would need to recompile the voxl-vision-px4 library everytime I wanted to update the path.

Saving VIO ned position: As for saving the VIO local position ned, my first inclination was to make T_body_wrt_local.d in the vio_manager.c script globably accessible, then just save it in my path script so it would be only saving state when switched to offboard mode. However, I came across this line in the vio_manager which seems to setup a pipe for publishing/subscribing to the local pose data. Is there an example somewhere for how to integrate this? This way I imagine just subscribing to this in the path script.

Cheers,

Pawel

PawelJ

Just an update as I've tried the above with some issues. I first tried to just compile and load the voxl-vision-px4 package from the repo following the README and had a few issues. A few key details: I am on ubuntu 20.04 and I cloned the repo to that machine.

Resolved Problems

After initializing the voxl-docker env on the Ubuntu machine, I had to update the CMakeLists.txt as it was using locations on my machine, and not the /home/root/ folders as expected once on the voxl
During the build process I had collected a few issues for voxl-mpa-tools and libmodal-json not being found. I followed the steps from this forum post using opkg, and after that I was able to build and run ./make_package.

Ongoing Problems
From that point I was able to move the file over with adb without an issue.

Right now I can run the voxl-test-vision-lib -o and get a reasonable xyz output while I manually move the drone around. However, I am still running into two problems.

my radio transmitter still connects to QGC without a problem, but it cannot arm the motors in manual mode anymore, nor can it activate remote mode. I believe this may be related to the next point...
QGC does not show any xy data for local_position_ned . I've faced this problem before and can usually resolve it by reconfiguring voxl-vision with voxl-configure-vision-px4 -f and just resetting the config to my ip afterwards, but no luck this time.

I came across this forum post that had a similar issue. If I check the status of voxl-qvio-server I get

~ # systemctl status voxl-qvio-server
● voxl-qvio-server.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

When I try to install it opkg as before I get this error about breaking dependencies

~ # opkg install voxl-qvio-server
Installing voxl-qvio-server (0.2.6) on root.
Not selecting libmodal_pipe 1.8.1 as installing it would break existing dependencies.
Not selecting libmodal_pipe 1.7.9 as installing it would break existing dependencies.
Not selecting libmodal_pipe 1.8.1 as installing it would break existing dependencies.
Collected errors:
 * satisfy_dependencies_for: Cannot satisfy the following dependencies for voxl-qvio-server:
 * 	libmodal_pipe (>= 1.4.1) * 
 * opkg_install_cmd: Cannot install package voxl-qvio-server.

This makes me think the original opkg commands I had to use to get voxl-vision-px4 to build installed some conflicting versions, however I am just guessing at the moment. In case it helps, here is a list of versions of the modal and voxl packages installed

~ # opkg list-installed | grep "modal"
libmodal_json - 0.3.4
libmodal_pipe - 1.8.1
modalai-vl - 0.1.3
~ # opkg list-installed | grep "voxl"
libvoxl_io - 0.5.4
voxl-cam-manager - 0.2.2
voxl-docker-support - 1.1.1
voxl-hal3-tof-cam-ros - 0.0.2
voxl-modem - 0.10.0
voxl-mpa-tools - 0.2.0
voxl-nodes - 0.0.8
voxl-rtsp - 1.0.2
voxl-suite - 0.2.0
voxl-time-sync - 0.0.1
voxl-utils - 0.5.2
voxl-vision-px4 - 0.8.1
voxl-vpn - 0.0.2
voxl_imu - 0.0.4

Any insight to this problem is greatly appreciated.

Hi Pawel,

I believe that the problem you're encountering is due to your installed version of voxl-suite (0.2.0). The older versions of voxl-suite were meant to be a snapshot of the voxl software bundle at a given point in time, so it had hard dependencies of the exact versions of packages that it wanted. We kept running into similar dependency issues as the one that you encountered, so all versions of voxl-suite > 0.2.0 will allow for updating packages. If you update your voxl-suite to the latest version (0.3.4 on stable, 0.4.1 on dev), you should be able to install voxl-qvio-server without issue.

Additionally, if you look at the dev branch of voxl-vision-px4, you'll see an offboard trajectory mode that allows you to feed vvpx4 a n<=10th dimensional polynomial for the vehicle to follow. This is our current solution to customizing offboard mode paths without having to mess with px4. It's still in the dev branch and will be seeing more testing and refining in the coming weeks in our aviary, but just a heads up that we will have support for customizing offboard paths in the near future.

Let me know what issues persist after the voxl-suite update

PawelJ

Thanks @modalAG for the quick reply. I tried updating to voxl-suite==0.3.4 and got an error saying I would need to update my system image from 2.5.2 to 3.2.0. I copied the terminal printout with relevant information below.

/ # opkg list-installed | grep "voxl"
libvoxl_io - 0.5.4
voxl-cam-manager - 0.2.2
voxl-docker-support - 1.1.1
voxl-hal3-tof-cam-ros - 0.0.2
voxl-modem - 0.10.0
voxl-mpa-tools - 0.2.0
voxl-nodes - 0.0.8
voxl-rtsp - 1.0.2
voxl-suite - 0.2.0
voxl-time-sync - 0.0.1
voxl-utils - 0.5.2
voxl-vision-px4 - 0.8.1
voxl-vpn - 0.0.2
voxl_imu - 0.0.4
/ # voxl-version
--------------------------------------------------------------------------------
system-image:    ModalAI 2.5.2 BUILDER: ekatzfey BUILD_TIME: 2020-08-24_00:08
kernel:          #1 SMP PREEMPT Wed Sep 9 22:28:09 UTC 2020 3.18.71-perf
factory-bundle:  1.0.1
--------------------------------------------------------------------------------
architecture:    aarch64
processor:       apq8096
os:              GNU/Linux
--------------------------------------------------------------------------------
voxl-suite:
Package: voxl-suite
Version: 0.3.4
Depends: voxl-utils (>= 0.6.0), libmodal_json (>= 0.3.4), libmodal_pipe (>= 1.7.8), librc_math (>= 1.1.4), libvoxl_io (>= 0.5.4), mavlink-camera-manager (>= 0.0.1), opencv (>= 4.5.1), openmp (>= 10.0.1), voxl-camera-server (>= 0.5.6), voxl-dfs-server (>= 0.0.7), voxl-docker-support (>= 1.1.1), voxl-gphoto2 (>= 0.0.5), voxl-hal3-tof-cam-ros (>= 0.0.5), voxl-imu-server (>= 0.7.8), voxl-modem (>= 0.11.0), voxl-mpa-tflite-server (>= 0.0.2), voxl-mpa-tools (>= 0.1.6), voxl-nodes (>= 0.1.3), voxl-qvio-server (>= 0.2.1), voxl-rtsp (>= 1.0.3), voxl-streamer (>= 0.2.1), voxl-vision-px4 (>= 0.8.1), voxl-vpn (>= 0.0.3)
Status: unknown ok not-installed
Section: base
Architecture: all
Maintainer: james@modalai.com
MD5Sum: e1e5bf1a125affe5036099f6ce895bed
Size: 1818
Filename: voxl-suite_0.3.4.ipk
Description: meta-package for voxl-suite stable release

Package: voxl-suite
Version: 0.2.0
Depends: docker, imu_app (= 0.0.6), libvoxl_io (= 0.5.2), voxl-cam-manager (= 0.2.2), voxl-docker-support (= 1.1.1), voxl-hal3-tof-cam-ros (= 0.0.2), voxl-modem (= 0.10.0), voxl-nodes (= 0.0.8), voxl-rtsp (= 1.0.2), voxl-utils (= 0.5.2), voxl-vision-px4 (= 0.6.8), voxl_imu (= 0.0.4), voxl-time-sync (= 0.0.1), voxl-vpn (= 0.0.2), librc_math (= 1.1.2), libmodal_pipe (= 1.2.2), modalai-vl (= 0.1.3)
Status: install user installed
Section: base
Architecture: armv7a
Maintainer: james@modalai.com
MD5Sum: af706cd3c1ea59f274f2ed9b93141f1d
Size: 870
Filename: voxl-suite_0.2.0.ipk
Description: meta-package to install all of the voxl-suite
Installed-Time: 199

Package: voxl-suite
Version: 0.4.1
Depends: voxl-utils (>= 0.6.1), libmodal_json (>= 0.3.4), libmodal_pipe (>= 1.7.8), librc_math (>= 1.1.4), libvoxl_io (>= 0.5.4), mavlink-camera-manager (>= 0.0.1), opencv (>= 4.5.1), openmp (>= 10.0.1), voxl-camera-server (>= 0.5.6), voxl-cpu-monitor (>= 0.1.1), voxl-dfs-server (>= 0.0.7), voxl-docker-support (>= 1.1.1), voxl-gphoto2 (>= 0.0.5), voxl-hal3-tof-cam-ros (>= 0.0.5), voxl-imu-server (>= 0.7.8), voxl-modem (>= 0.11.0), voxl-mpa-tflite-server (>= 0.0.2), voxl-mpa-tools (>= 0.1.6), voxl-nodes (>= 0.1.3), voxl-qvio-server (>= 0.2.1), voxl-rtsp (>= 1.0.3), voxl-streamer (>= 0.2.1), voxl-vision-px4 (>= 0.8.1), voxl-vpn (>= 0.0.3)
Replaces: voxl-suite-dev
Status: unknown ok not-installed
Section: base
Architecture: all
Maintainer: james@modalai.com
MD5Sum: 5a4532bdaa4ac5bf00cf64f83cd05c38
Size: 1848
Filename: voxl-suite_0.4.1_202104021902.ipk
Description: meta-package for voxl-suite software collection

--------------------------------------------------------------------------------
/ # opkg update
Downloading http://voxl-packages.modalai.com/stable/Packages.gz.
Updated source 'stable'.
Downloading http://voxl-packages.modalai.com/dev/Packages.gz.
Updated source 'dev'.
Downloading file:///home/root/voxl-suite-ipk/Packages.gz.
Updated source 'local'.
/ # opkg install voxl-suite
Upgrading voxl-suite from 0.2.0 to 0.4.1 on root.
Collected errors:
 * satisfy_dependencies_for: Cannot satisfy the following dependencies for voxl-suite:
 * 	libtof-interface (>= 0.0.2) * 	libtof-interface (>= 0.0.2) * 	royale-331-spectre-4-7 (>= 0.0.3) * 
 * opkg_install_cmd: Cannot install package voxl-suite.
/ # opkg install voxl-suite --force-depends
Upgrading voxl-suite from 0.2.0 to 0.4.1 on root.
Upgrading voxl-utils from 0.5.2 to 0.6.4 on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-utils_0.6.4_202105042150.ipk.
Installing mavlink-camera-manager (0.0.2) on root.
Downloading http://voxl-packages.modalai.com/dev/mavlink-camera-manager_0.0.2_202103020300.ipk.
Installing voxl-camera-server (0.6.4) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-camera-server_0.6.4_202104061841.ipk.
Installing libmodal_exposure (0.0.2) on root.
Downloading http://voxl-packages.modalai.com/dev/libmodal_exposure_0.0.2_202104052336.ipk.
Installing voxl-cpu-monitor (0.1.5) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-cpu-monitor_0.1.5_202105050004.ipk.
Installing voxl-dfs-server (0.0.7) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-dfs-server_0.0.7_202102282128.ipk.
Installing voxl-gphoto2 (0.0.5) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-gphoto2_0.0.5_202103020119.ipk.
Upgrading voxl-hal3-tof-cam-ros from 0.0.2 to 0.0.5 on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-hal3-tof-cam-ros_0.0.5_202103011803.ipk.
Installing voxl-imu-server (0.7.9) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-imu-server_0.7.9_202104130019.ipk.
Upgrading voxl-modem from 0.10.0 to 0.11.0 on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-modem_0.11.0_202102270054.ipk.
Installing voxl-mpa-tflite-server (0.0.2) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-mpa-tflite-server_0.0.2_202104162223.ipk.
Upgrading voxl-nodes from 0.0.8 to 0.1.3 on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-nodes_0.1.3_202103222000.ipk.
Removing obsolete file /opt/ros/indigo/share/snap_imu/cmake/snap_imuConfig-version.cmake.
Removing obsolete file /opt/ros/indigo/lib/libsnap_imu.so.
Removing obsolete file /opt/ros/indigo/lib/libsnap_imu_nodelet.so.
Removing obsolete file /opt/ros/indigo/share/snap_imu/launch/imu.launch.
Removing obsolete file /opt/ros/indigo/lib/libSnapdragonImuManager.so.
Removing obsolete file /opt/ros/indigo/share/snap_imu/cmake/snap_imuConfig.cmake.
Removing obsolete file /opt/ros/indigo/lib/pkgconfig/snap_imu.pc.
Removing obsolete file /opt/ros/indigo/lib/snap_imu/snap_imu_node.
Removing obsolete file /opt/ros/indigo/share/snap_imu/package.xml.
Removing obsolete file /opt/ros/indigo/share/snap_imu/snap_imu_nodelet.xml.
Installing voxl-qvio-server (0.2.6) on root.
Upgrading voxl-rtsp from 1.0.2 to 1.0.4 on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-rtsp_1.0.4_202104280317.ipk.
Installing voxl-streamer (0.2.1) on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-streamer_0.2.1_202103020021.ipk.
Upgrading voxl-vpn from 0.0.2 to 0.0.3 on root.
Downloading http://voxl-packages.modalai.com/dev/voxl-vpn_0.0.3_202102050123.ipk.
voxl-cam-manager was autoinstalled and is now orphaned, removing.
Removing package voxl-cam-manager from root...
voxl_imu was autoinstalled and is now orphaned, removing.
Removing package voxl_imu from root...
voxl-time-sync was autoinstalled and is now orphaned, removing.
Removing package voxl-time-sync from root...
voxl-suite installer detected system image 2.5.2

ERROR, this version of voxl-suite requires system image>=3.2.0
if you know what you are doing, you can continue the install anyway
would you like to continue or quit?
1) continue
2) quit
#? 2

I followed the steps in the docs here, to install VOXL Platform Release 3.2.0-0.0.2 from this link. I chose to retain the /data/ partition, but am waiting for the voxl to reboot. The documentation says this should take ~3min and I have been waiting for ~15min. Would it be possible to get on a video call to sort through this? I would hate to brick the system, if it isn't already.

Chad Sweet

Can you send the screenshot from the flashing process? Sometimes you need to run with 'sudo' or manually install via 'sudo fastboot'

if you type 'sudo fastboot devices' what do you see?

PawelJ

Hi @Chad-Sweet ,

Thanks for replying so quickly, I really appreciate it as this has been blocking for my work.

When I check for fastboot devices I see

(test) pjaworsk@ABR04:~/src/voxl/voxl-vision-px4$ sudo fastboot devices
[sudo] password for pjaworsk: 
534625ca	fastboot

modaltb

In the installer folder that gets unzipped, there's a system-image directory, which has a flash_build_apps.shscript.

This is getting stuck on:

while [ "$(fastboot devices)" == "" ]
	do
		echo "[INFO] Waiting for fastboot..."
		sleep 2
	done

Can you please try to run the installer with sudo? e.g. sudo ./install.sh

Something with permissions can get funky with fastboot and running with sudo seems to get through the issue normally.

PawelJ

Thanks, that got the install through. You can ignore the panicked email I sent out earlier I was worried I had bricked the device and would have to wait for a usb expansion board to ship.

That fixed the issue with VIO local position ned and it is now visible in QGC again. I still seem to have the issue with my radio transmitter. QGC vocalizes the commands as usual, but I can't seem to arm the motors. When I go to the radio page of QGC it gives me this error Screenshot from 2021-05-05 12-59-57.png
It looks like this could have to do with the calibration going off the docs, but if I try to recalibrate the same error comes up.

PawelJ

It looks like I spoke too soon. The VIO data was sent after the initial reinstall. Once I tried compiling and building the dev branch of voxl-vision-px4 (to use the polynomial path planning) it stopped sending to QGC again. I tried installing the master branch to fix it, but it still isn't sending. Following the same steps I listed above that I followed from this forum post, The voxl-test-vision-lib script functions as expected and voxl-vision-px4 voxl-camera-server and voxl-qvio-server are running.

~ # systemctl status voxl-vision-px4
● voxl-vision-px4.service - voxl-vision-px4
   Loaded: loaded (/usr/bin/voxl-vision-px4; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-05 19:45:52 UTC; 14s ago
  Process: 11248 ExecStartPre=/bin/sleep 2 (code=exited, status=0/SUCCESS)
 Main PID: 11259 (voxl-vision-px4)
   CGroup: /system.slice/voxl-vision-px4.service
           └─11259 /usr/bin/voxl-vision-px4

May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting fixed pose input
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting vio manager
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting apriltag manager
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting voa manager
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting offboard figure eight
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Init complete, entering main loop
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Connected to voxl-qvio-server
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: done updating transforms to use imu: imu1
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Added new UDP connection to 192.168.86.176
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: PX4 Connected over UART with sysid 1

~ # systemctl status voxl-qvio-server
● voxl-qvio-server.service - voxl-qvio-server
   Loaded: loaded (/usr/bin/voxl-qvio-server; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-05 19:23:52 UTC; 22min ago
 Main PID: 6081 (voxl-qvio-serve)
   CGroup: /system.slice/voxl-qvio-server.service
           └─6081 /usr/bin/voxl-qvio-server

May 05 19:23:52 apq8096 voxl-qvio-server[6081]: LNX_IA64 supported? 1
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: WINDOWS supported? 0
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: AR ERROR: arFileOpen(): Failed to open file: //vislam/Configu....xml
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: FASTCV: fcvAvailableHardware Linux
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: mempool cur block size 307200, new block size 307200
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: Please ignore the error about Configuration.SF.xml above. ^^^
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: It's an optional file, and should be a warning not an error
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: waiting for imu
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: waiting for cam
May 05 19:44:32 apq8096 voxl-qvio-server[6081]: WARNING: output data thread fell behind
Hint: Some lines were ellipsized, use -l to show in full.

~ # systemctl status voxl-camera-server
● voxl-camera-server.service - voxl-camera-server
   Loaded: loaded (/usr/bin/voxl-camera-server; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-05 19:45:33 UTC; 56s ago
 Main PID: 10989 (voxl-camera-ser)
   CGroup: /system.slice/voxl-camera-server.service
           └─10989 /usr/bin/voxl-camera-server -c /etc/modalai/voxl-camera-server.conf

May 05 19:45:34 apq8096 bash[10989]: AEAlgo     : mvcpa
May 05 19:45:34 apq8096 bash[10989]: mvcpa_filter_size    : 2
May 05 19:45:34 apq8096 bash[10989]: mvcpa_exp_cost       : 0.7500
May 05 19:45:34 apq8096 bash[10989]: mvcpa_gain_cost      : 0.2500
May 05 19:45:34 apq8096 bash[10989]: mvcpa_histogram      : false
May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/hires_preview/ channel: 0
May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/stereo/ channel: 1
May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/tracking/ channel: 2
May 05 19:45:34 apq8096 bash[10989]: tracking
May 05 19:45:34 apq8096 bash[10989]: Camera Width: 640, Height: 480, Format: 291 not supported!

However, my voxl-imu-server status is different, and I'm not sure if this is as it's expected to be. It also remains like this after I restart the service

~ # systemctl status voxl-imu-server
● voxl-imu-server.service - voxl-imu-server
   Loaded: loaded (/usr/bin/voxl-imu-server; enabled; vendor preset: enabled)
   Active: activating (start-pre) since Wed 2021-05-05 19:49:03 UTC; 1s ago
  Process: 11969 ExecStart=/usr/bin/voxl-imu-server (code=exited, status=255)
 Main PID: 11969 (code=exited, status=255);         : 11975 (sleep)
   CGroup: /system.slice/voxl-imu-server.service
           └─control
             └─11975 /bin/sleep 2

I've included a list of my voxl and modal packages since the update

/ # opkg list-installed | grep "voxl"
libvoxl_io - 0.5.4
voxl-camera-server - 0.5.7
voxl-dfs-server - 0.0.7
voxl-docker-support - 1.1.1
voxl-gphoto2 - 0.0.5
voxl-hal3-tof-cam-ros - 0.0.5
voxl-imu-server - 0.7.8
voxl-modem - 0.11.0
voxl-mpa-tflite-server - 0.0.2
voxl-mpa-tools - 0.1.6
voxl-nodes - 0.1.3
voxl-qvio-server - 0.2.1
voxl-rtsp - 1.0.3
voxl-streamer - 0.2.1
voxl-suite - 0.3.4
voxl-utils - 0.6.0
voxl-vision-px4 - 0.8.1
voxl-vpn - 0.0.3
/ # opkg list-installed | grep "modal"
libmodal_json - 0.3.4
libmodal_pipe - 1.7.9
modalai-vl - 0.1.3

Just for completeness, I've also included my voxl-vision-px4.conf

/**
 * VOXL Vision PX4 Configuration File
 *
 */
{
        "qgc_ip":       "192.168.86.208",
        "en_localhost_mavlink_udp":     true,
        "en_secondary_qgc":     false,
        "secondary_qgc_ip":     "192.168.1.214",
        "qgc_udp_port_number":  14550,
        "localhost_udp_port_number":    14551,
        "udp_mtu":      512,
        "en_vio":       true,
        "en_voa":       true,
        "en_send_vio_to_qgc":   true,
        "en_send_voa_to_qgc":   false,
        "en_set_clock_from_gps":        true,
        "en_force_onboard_mav1_mode":   true,
        "en_reset_px4_on_error":        true,
        "qvio_auto_reset_quality":      0.00050000002374872565,
        "en_adsb":      false,
        "adsb_uart_bus":        7,
        "adsb_uart_baudrate":   57600,
        "px4_uart_bus": 5,
        "px4_uart_baudrate":    921600,
        "offboard_mode":        "figure_eight",
        "follow_tag_id":        0,
        "en_apriltag_fixed_frame":      false,
        "fixed_frame_filter_len":       5,
        "en_transform_mavlink_pos_setpoints_from_fixed_frame":  false
}

PawelJ

I've sent a few messages and listed a few problems above, so I just want to reiterate the current problem that's lead me here to stay on track.

I'm trying to set a custom path and record the VIO position output (local_position_ned) from the named pipe while the path is being followed. My plan is to use the dev branch of voxl-vision-px4 and incorporate something like this example from libmodal-pipe to do the reading. This would be incorporated into the path script so it gets run whenever the offboard mode is following the path. I still have to look into the polynomial path planning code so can't speak on that yet.

For the use case I imagine I would still have manual control from the transmitter so I can manual set the kill switch and take over control if anything happens. I also use this to enable offboard mode. Given the problems above, I can't use the transmitter to set the mode on the drone, so I can't test out any offboard flight. Since I'm also embedding the vio feedback recording in that path, I can't run that without switching to offboard mode.

I imagine the issue of VIO not being sent to QGC is related to the transmitter communicating with QGC, but not with the drone (no state change or motor arming).

Hi Pawel,
It's hard to tell exactly what's going on from these cutouts, would you be available for a short 30ish minute call where you can explain exactly what the issue is and I can help real time to figure out what's wrong, I think it'll be much quicker than going back and forth here.

PawelJ

Hi @Alex-Gardner ,

That would be great! We can do it over a video chat if that works. Feel free to send me a link at my email pawel.jaworski@appliedbrainresearch.com.

Thank you

PawelJ

Just posting an update after some discuission with @Alex-Gardner. Thank you again, you were incredibly helpful in understanding the overall voxl comm pipeline.

Solution for Python I/O
The easiest method to do this was to run a modified vvpx4 that would record vio position based on a flag. Using the dev branch, I modified process_new_vio_data() in vio_manager found here. I just added a snippet that would write T_body_wrt_local.d to a file in /data/my_folder. I have the flag set to start and stop logging when an offboard flight mode is started and stopped, by setting a flag in the HOME block of the path.

Solution for QGroundControl and Transmitter Connection Problems
I had to reset the px4 parameters and recalibrate. This fixed my connectivity issues and I can now change modes and arm the motors through my transmitter. I can also see VIO data in QGC as well. The process I followed for this portion was based off the solution to this forum post and the steps in the docs here:

reset px4 parameters to firmare's default
load the m500_4s_config.params - the instructions say to do this twice, however I think this is outdated as it didn't seem necessary for me, as the second time I went to upload the params I got a "no changes made" type notification
load the mavlink_serial_ports.params helper config
run through the sensor calibration steps on QGC, I used a flat sheet of mdf on a large turntable to keep rotate the drone. Standing on its 4 legs and upside-down was fine, but for the other 4 orientations I would balance it on two legs. It would stay stable, but would be slightly off from a perfect right angle. From the PX4 docs it looks like this is fine for accelerometer calibration since it uses a least squares fit.

Unresolved Problems
There is still something wrong here based on how the drone is flying, and I cannot tell whether it's related to the calibration or some missing parameters. It mostly flies fine in manual mode, maybe a little bit of drift, but it seems about the same as it was when I received the drone calibrated. In position control it starts to drift when in flight, and when stationary the VIO output drifts as well. Here is a video I took of it "in the field." Appologies for the cellphone video, I had no way of screencapturing at the time. You can see the drone is stationary and it's still drifting. I don't think it has to do with trackable features, as this setup worked with when I first got the drone. I also took a video of the flight, going from manual, to position, to offboard (I say in the video when I change modes). You can see the drift here as I try to keep the drone centered in the net. The vio drift video with the drone stationary is with my modified vvpx4 where the vio position is saved when in offboard mode, so this should make no changes. However, I also tested with the master branch of vvpx4, which is the second video with the drone flying. Since it happens on the master branch, I think the problem is related to the QGC/PX4 setup and calibration.

Q1 One possible problem is that the compass calibration says to stay away from metal, so possibly the 12" turntable was causing some interference, but I don't think it is likely because of the thick mdf on top of it. One point I am unsure of is whether the gps is required for this, as where I calibrate has no gps signal. Can you confirm whether gps is required for this step (I imagine not since you offer a gps free version)?

Q2 One other possible problem is that when I look at my px4 parameters, I do not see the PWM_MIN parameter listed in the docs when testing if the parameters were uploaded, which could be a sign of some missing px4 configs. From a fresh QGC/PX4 setup, is there something other than the mavlink_serial_ports and m500_4s_config that is required?

PawelJ

I reran the calibration outdoors with gps signal and far from any metal. I also raised the drone to be ~3-4' from the metal turntable below it. This improved the calibration quite a bit and the drone is not drifting as much as before. However, there still seems to be more drift in the VIO than there was when I first received the drone. I still don't have the PWM_MIN parameter that the docs say I should check for the assure the PX4 parameters loaded properly. I see that this parameter is in the custom2.params file, but the note in the document says it turns off the gps.

Can you please explain the steps you go through during your calibration process? Should I also be loading this custom2.params file? If so, does that mean that gps is not used when in VIO (no sensor fusion etc)?