# I/O Python Interfacing Source: https://forum.modalai.com/topic/204/i-o-python-interfacing Category: Development Drones (https://forum.modalai.com/category/44/development-drones) Tags: m500 Posted: 2021-04-20 17:50:49 UTC by PawelJ Replies: 18 · Views: 3789 ## PawelJ · 2021-04-20 17:50:49 UTC Hello, I have been going through the documentation for some more details on how to develop your own controllers and path planners in python, but there are quite a few sections and they mainly focus on the build process. What would be the recommended way to run python scripts on the voxl-m500? I would mainly be interested in getting state and image feedback, and sending in target information. For the controller, I would like to keep what is already running, as it works great, but would like to add an adaptive controller ontop of that that can be added to the signal being sent to the motors (whether it's speed control, pwm, etc). Lastly, will these scripts be run from a computer ssh'd into the drone, or is this something that is onboard and will be triggered when in a certain flight mode? Any guidance or example scripts are greatly appreciated. Cheers, Pawel J ## Reply by modaltb (ModalAI staff) · 2021-04-22 15:46:42 UTC Hi @PawelJ , If you're comfortable with Docker, here's an approach most folks seem to be using in some form to get the 'base system' going: - Use an Ubuntu/Alpine Docker container on the VOXL-M500 (the M500 has the VOXL companion computer, this runs a Yocto linux, which is not as easy as Ubuntu for getting packages, so getting a container on there is an easy way to get your environment setup. Here's some docs: https://docs.modalai.com/docker-on-voxl/ - The Docker container can have whatever Python you want in there. Here's an example of how we have a Python MAVSDK container: https://gitlab.com/voxl-public/voxl-docker-images/voxl-docker-mavsdk-python - VOXL is running Linux, so you can make things run on bootup using systemd etc. Here's how: https://docs.modalai.com/docker-on-voxl/#configuring-a-program-to-run-automatically-inside-a-docker-on-boot You could also ssh over and run the container manually, or even run a web server in the container and expose an API ;) ## Reply by PawelJ · 2021-04-22 19:20:53 UTC Hi @modaltb , Thanks for the breakdown this is very helpful. I'll have to go through those documents :ok_hand: The web server is a very intriguing idea. Do you have any idea what kind of communication frequency you can get with that? Also, can you expand on what you mean by `base system`? Do you have a python api that can be used to access the modalAI functions for VIO, control, obstacle avoidance etc? Thanks again, I have really appreciated the quick reponses from the dev team. ## Reply by modaltb (ModalAI staff) · 2021-04-22 23:01:49 UTC We don't have a Python interface for much yet... basically an I2C interface for now, but it's slowly being added. OK jumping around a little bit, but we do a lot in C, and a lot through named pipes most recently. New docs are coming online in the next couple weeks, but the source code is up on https://gitlab.com/voxl-public This is a pretty clean way for getting sensor data (VIO, IMUs, etc.), via a named pipe (basically you could probably do a simple Python 'read' on a named pipe and consume use the data for what you need. Controlling of the Flight Controller is a bit different, it's 'outside' of VOXL basically on it's own microcontroller, and interfaced via MAVLink, and handled by PX4. So you're bound to that interface (MAVLink) and features (PX4). Not sure the rates on webserver, MAVLink to a Ground Station is likely easier when I think about it.... but there's a lot of options depending on the dev stack you're comfortable with. ## Reply by PawelJ · 2021-04-23 13:01:10 UTC Thank you for the abundance of helpful information :) I'll have to parse through all of this, but the road map helps. Cheers, Pawel ## Reply by PawelJ · 2021-05-04 15:31:25 UTC Hi @modaltb , I've been going through the documentation and testing various methods based on the steps you laid out above. Right now I am just focussing on setting my own path for offboard mode, and recording the VIO local_position_ned outputs to file while it is flying that path. I was looking at the mavsdk-python package at first, but If I'm following the code in the repo correctly, it looks like it will be easier to modify and recompile the voxl-vision-px4 package. It also looks like this option will have faster communication (uart over udp?). I was hoping you could let me know if the plan of action makes sense, or if there is a simpler / more efficient way to do so. **Custom Path**: Referencing the [voxl-vision-px4 repo](https://gitlab.com/voxl-public/modal-pipe-architecture/voxl-vision-px4/-/tree/master), It looks like I can add my own path generation script, similar to `offboard_figure_eight.c` and add it as an option in `offboard_mode.c`. Then I can just update the `offboard mode` in `/etc/modalai/voxl-vision-px4.conf` to my custom offboard path. The downside to this is I would need to recompile the voxl-vision-px4 library everytime I wanted to update the path. **Saving VIO ned position**: As for saving the VIO local position ned, my first inclination was to make `T_body_wrt_local.d` in the `vio_manager.c` script globably accessible, then just save it in my path script so it would be only saving state when switched to `offboard mode`. However, I came across [this line](https://gitlab.com/voxl-public/modal-pipe-architecture/voxl-vision-px4/-/blob/master/src/vio_manager.c#L200) in the `vio_manager` which seems to setup a pipe for publishing/subscribing to the local pose data. Is there an example somewhere for how to integrate this? This way I imagine just subscribing to this in the path script. Cheers, Pawel ## Reply by PawelJ · 2021-05-04 20:35:35 UTC Just an update as I've tried the above with some issues. I first tried to just compile and load the voxl-vision-px4 package from the repo following the README and had a few issues. A few key details: I am on ubuntu 20.04 and I cloned the repo to that machine. **Resolved Problems** - After initializing the voxl-docker env on the Ubuntu machine, I had to update the CMakeLists.txt as it was using locations on my machine, and not the `/home/root/` folders as expected once on the voxl - During the build process I had collected a few issues for `voxl-mpa-tools` and `libmodal-json` not being found. I followed the steps from [this forum post](https://forum.modalai.com/topic/129/error-compiling-voxl-vision-px4-from-source/8?_=1620147287855) using `opkg`, and after that I was able to build and run `./make_package`. **Ongoing Problems** From that point I was able to move the file over with adb without an issue. Right now I can run the `voxl-test-vision-lib -o` and get a reasonable xyz output while I manually move the drone around. However, I am still running into two problems. 1. my radio transmitter still connects to QGC without a problem, but it cannot arm the motors in manual mode anymore, nor can it activate remote mode. I believe this may be related to the next point... 2. QGC does not show any xy data for `local_position_ned` . I've faced this problem before and can usually [resolve it by reconfiguring voxl-vision](https://forum.modalai.com/topic/76/help-lossing-data-from-camera-imu-tracking/6) with `voxl-configure-vision-px4 -f` and just resetting the config to my ip afterwards, but no luck this time. I came across [this forum post](https://forum.modalai.com/topic/140/vio-indoor-navigation/9?_=1620147287892) that had a similar issue. If I check the status of `voxl-qvio-server` I get ``` ~ # systemctl status voxl-qvio-server ● voxl-qvio-server.service Loaded: not-found (Reason: No such file or directory) Active: inactive (dead) ``` When I try to install it `opkg` as before I get this error about breaking dependencies ``` ~ # opkg install voxl-qvio-server Installing voxl-qvio-server (0.2.6) on root. Not selecting libmodal_pipe 1.8.1 as installing it would break existing dependencies. Not selecting libmodal_pipe 1.7.9 as installing it would break existing dependencies. Not selecting libmodal_pipe 1.8.1 as installing it would break existing dependencies. Collected errors: * satisfy_dependencies_for: Cannot satisfy the following dependencies for voxl-qvio-server: * libmodal_pipe (>= 1.4.1) * * opkg_install_cmd: Cannot install package voxl-qvio-server. ``` This makes me think the original `opkg` commands I had to use to get `voxl-vision-px4` to build installed some conflicting versions, however I am just guessing at the moment. In case it helps, here is a list of versions of the `modal` and `voxl` packages installed ``` ~ # opkg list-installed | grep "modal" libmodal_json - 0.3.4 libmodal_pipe - 1.8.1 modalai-vl - 0.1.3 ~ # opkg list-installed | grep "voxl" libvoxl_io - 0.5.4 voxl-cam-manager - 0.2.2 voxl-docker-support - 1.1.1 voxl-hal3-tof-cam-ros - 0.0.2 voxl-modem - 0.10.0 voxl-mpa-tools - 0.2.0 voxl-nodes - 0.0.8 voxl-rtsp - 1.0.2 voxl-suite - 0.2.0 voxl-time-sync - 0.0.1 voxl-utils - 0.5.2 voxl-vision-px4 - 0.8.1 voxl-vpn - 0.0.2 voxl_imu - 0.0.4 ``` Any insight to this problem is greatly appreciated. ## Reply by Guest · 2021-05-05 00:27:17 UTC Hi Pawel, I believe that the problem you're encountering is due to your installed version of voxl-suite (0.2.0). The older versions of voxl-suite were meant to be a snapshot of the voxl software bundle at a given point in time, so it had hard dependencies of the exact versions of packages that it wanted. We kept running into similar dependency issues as the one that you encountered, so all versions of voxl-suite > 0.2.0 will allow for updating packages. If you update your voxl-suite to the latest version (0.3.4 on stable, 0.4.1 on dev), you should be able to install voxl-qvio-server without issue. Additionally, if you look at the dev branch of voxl-vision-px4, you'll see an offboard trajectory mode that allows you to feed vvpx4 a n<=10th dimensional polynomial for the vehicle to follow. This is our current solution to customizing offboard mode paths without having to mess with px4. It's still in the dev branch and will be seeing more testing and refining in the coming weeks in our aviary, but just a heads up that we will have support for customizing offboard paths in the near future. Let me know what issues persist after the voxl-suite update ## Reply by PawelJ · 2021-05-05 15:27:40 UTC Thanks @modalAG for the quick reply. I tried updating to `voxl-suite==0.3.4` and got an error saying I would need to update my system image from `2.5.2` to `3.2.0`. I copied the terminal printout with relevant information below. ``` / # opkg list-installed | grep "voxl" libvoxl_io - 0.5.4 voxl-cam-manager - 0.2.2 voxl-docker-support - 1.1.1 voxl-hal3-tof-cam-ros - 0.0.2 voxl-modem - 0.10.0 voxl-mpa-tools - 0.2.0 voxl-nodes - 0.0.8 voxl-rtsp - 1.0.2 voxl-suite - 0.2.0 voxl-time-sync - 0.0.1 voxl-utils - 0.5.2 voxl-vision-px4 - 0.8.1 voxl-vpn - 0.0.2 voxl_imu - 0.0.4 / # voxl-version -------------------------------------------------------------------------------- system-image: ModalAI 2.5.2 BUILDER: ekatzfey BUILD_TIME: 2020-08-24_00:08 kernel: #1 SMP PREEMPT Wed Sep 9 22:28:09 UTC 2020 3.18.71-perf factory-bundle: 1.0.1 -------------------------------------------------------------------------------- architecture: aarch64 processor: apq8096 os: GNU/Linux -------------------------------------------------------------------------------- voxl-suite: Package: voxl-suite Version: 0.3.4 Depends: voxl-utils (>= 0.6.0), libmodal_json (>= 0.3.4), libmodal_pipe (>= 1.7.8), librc_math (>= 1.1.4), libvoxl_io (>= 0.5.4), mavlink-camera-manager (>= 0.0.1), opencv (>= 4.5.1), openmp (>= 10.0.1), voxl-camera-server (>= 0.5.6), voxl-dfs-server (>= 0.0.7), voxl-docker-support (>= 1.1.1), voxl-gphoto2 (>= 0.0.5), voxl-hal3-tof-cam-ros (>= 0.0.5), voxl-imu-server (>= 0.7.8), voxl-modem (>= 0.11.0), voxl-mpa-tflite-server (>= 0.0.2), voxl-mpa-tools (>= 0.1.6), voxl-nodes (>= 0.1.3), voxl-qvio-server (>= 0.2.1), voxl-rtsp (>= 1.0.3), voxl-streamer (>= 0.2.1), voxl-vision-px4 (>= 0.8.1), voxl-vpn (>= 0.0.3) Status: unknown ok not-installed Section: base Architecture: all Maintainer: james@modalai.com MD5Sum: e1e5bf1a125affe5036099f6ce895bed Size: 1818 Filename: voxl-suite_0.3.4.ipk Description: meta-package for voxl-suite stable release Package: voxl-suite Version: 0.2.0 Depends: docker, imu_app (= 0.0.6), libvoxl_io (= 0.5.2), voxl-cam-manager (= 0.2.2), voxl-docker-support (= 1.1.1), voxl-hal3-tof-cam-ros (= 0.0.2), voxl-modem (= 0.10.0), voxl-nodes (= 0.0.8), voxl-rtsp (= 1.0.2), voxl-utils (= 0.5.2), voxl-vision-px4 (= 0.6.8), voxl_imu (= 0.0.4), voxl-time-sync (= 0.0.1), voxl-vpn (= 0.0.2), librc_math (= 1.1.2), libmodal_pipe (= 1.2.2), modalai-vl (= 0.1.3) Status: install user installed Section: base Architecture: armv7a Maintainer: james@modalai.com MD5Sum: af706cd3c1ea59f274f2ed9b93141f1d Size: 870 Filename: voxl-suite_0.2.0.ipk Description: meta-package to install all of the voxl-suite Installed-Time: 199 Package: voxl-suite Version: 0.4.1 Depends: voxl-utils (>= 0.6.1), libmodal_json (>= 0.3.4), libmodal_pipe (>= 1.7.8), librc_math (>= 1.1.4), libvoxl_io (>= 0.5.4), mavlink-camera-manager (>= 0.0.1), opencv (>= 4.5.1), openmp (>= 10.0.1), voxl-camera-server (>= 0.5.6), voxl-cpu-monitor (>= 0.1.1), voxl-dfs-server (>= 0.0.7), voxl-docker-support (>= 1.1.1), voxl-gphoto2 (>= 0.0.5), voxl-hal3-tof-cam-ros (>= 0.0.5), voxl-imu-server (>= 0.7.8), voxl-modem (>= 0.11.0), voxl-mpa-tflite-server (>= 0.0.2), voxl-mpa-tools (>= 0.1.6), voxl-nodes (>= 0.1.3), voxl-qvio-server (>= 0.2.1), voxl-rtsp (>= 1.0.3), voxl-streamer (>= 0.2.1), voxl-vision-px4 (>= 0.8.1), voxl-vpn (>= 0.0.3) Replaces: voxl-suite-dev Status: unknown ok not-installed Section: base Architecture: all Maintainer: james@modalai.com MD5Sum: 5a4532bdaa4ac5bf00cf64f83cd05c38 Size: 1848 Filename: voxl-suite_0.4.1_202104021902.ipk Description: meta-package for voxl-suite software collection -------------------------------------------------------------------------------- / # opkg update Downloading http://voxl-packages.modalai.com/stable/Packages.gz. Updated source 'stable'. Downloading http://voxl-packages.modalai.com/dev/Packages.gz. Updated source 'dev'. Downloading file:///home/root/voxl-suite-ipk/Packages.gz. Updated source 'local'. / # opkg install voxl-suite Upgrading voxl-suite from 0.2.0 to 0.4.1 on root. Collected errors: * satisfy_dependencies_for: Cannot satisfy the following dependencies for voxl-suite: * libtof-interface (>= 0.0.2) * libtof-interface (>= 0.0.2) * royale-331-spectre-4-7 (>= 0.0.3) * * opkg_install_cmd: Cannot install package voxl-suite. / # opkg install voxl-suite --force-depends Upgrading voxl-suite from 0.2.0 to 0.4.1 on root. Upgrading voxl-utils from 0.5.2 to 0.6.4 on root. Downloading http://voxl-packages.modalai.com/dev/voxl-utils_0.6.4_202105042150.ipk. Installing mavlink-camera-manager (0.0.2) on root. Downloading http://voxl-packages.modalai.com/dev/mavlink-camera-manager_0.0.2_202103020300.ipk. Installing voxl-camera-server (0.6.4) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-camera-server_0.6.4_202104061841.ipk. Installing libmodal_exposure (0.0.2) on root. Downloading http://voxl-packages.modalai.com/dev/libmodal_exposure_0.0.2_202104052336.ipk. Installing voxl-cpu-monitor (0.1.5) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-cpu-monitor_0.1.5_202105050004.ipk. Installing voxl-dfs-server (0.0.7) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-dfs-server_0.0.7_202102282128.ipk. Installing voxl-gphoto2 (0.0.5) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-gphoto2_0.0.5_202103020119.ipk. Upgrading voxl-hal3-tof-cam-ros from 0.0.2 to 0.0.5 on root. Downloading http://voxl-packages.modalai.com/dev/voxl-hal3-tof-cam-ros_0.0.5_202103011803.ipk. Installing voxl-imu-server (0.7.9) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-imu-server_0.7.9_202104130019.ipk. Upgrading voxl-modem from 0.10.0 to 0.11.0 on root. Downloading http://voxl-packages.modalai.com/dev/voxl-modem_0.11.0_202102270054.ipk. Installing voxl-mpa-tflite-server (0.0.2) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-mpa-tflite-server_0.0.2_202104162223.ipk. Upgrading voxl-nodes from 0.0.8 to 0.1.3 on root. Downloading http://voxl-packages.modalai.com/dev/voxl-nodes_0.1.3_202103222000.ipk. Removing obsolete file /opt/ros/indigo/share/snap_imu/cmake/snap_imuConfig-version.cmake. Removing obsolete file /opt/ros/indigo/lib/libsnap_imu.so. Removing obsolete file /opt/ros/indigo/lib/libsnap_imu_nodelet.so. Removing obsolete file /opt/ros/indigo/share/snap_imu/launch/imu.launch. Removing obsolete file /opt/ros/indigo/lib/libSnapdragonImuManager.so. Removing obsolete file /opt/ros/indigo/share/snap_imu/cmake/snap_imuConfig.cmake. Removing obsolete file /opt/ros/indigo/lib/pkgconfig/snap_imu.pc. Removing obsolete file /opt/ros/indigo/lib/snap_imu/snap_imu_node. Removing obsolete file /opt/ros/indigo/share/snap_imu/package.xml. Removing obsolete file /opt/ros/indigo/share/snap_imu/snap_imu_nodelet.xml. Installing voxl-qvio-server (0.2.6) on root. Upgrading voxl-rtsp from 1.0.2 to 1.0.4 on root. Downloading http://voxl-packages.modalai.com/dev/voxl-rtsp_1.0.4_202104280317.ipk. Installing voxl-streamer (0.2.1) on root. Downloading http://voxl-packages.modalai.com/dev/voxl-streamer_0.2.1_202103020021.ipk. Upgrading voxl-vpn from 0.0.2 to 0.0.3 on root. Downloading http://voxl-packages.modalai.com/dev/voxl-vpn_0.0.3_202102050123.ipk. voxl-cam-manager was autoinstalled and is now orphaned, removing. Removing package voxl-cam-manager from root... voxl_imu was autoinstalled and is now orphaned, removing. Removing package voxl_imu from root... voxl-time-sync was autoinstalled and is now orphaned, removing. Removing package voxl-time-sync from root... voxl-suite installer detected system image 2.5.2 ERROR, this version of voxl-suite requires system image>=3.2.0 if you know what you are doing, you can continue the install anyway would you like to continue or quit? 1) continue 2) quit #? 2 ``` I followed the steps [in the docs here](https://docs.modalai.com/flash-system-image/), to install `VOXL Platform Release 3.2.0-0.0.2` from [this link](https://developer.modalai.com/asset). I chose to retain the `/data/` partition, but am waiting for the voxl to reboot. The documentation says this should take ~3min and I have been waiting for ~15min. Would it be possible to get on a video call to sort through this? I would hate to brick the system, if it isn't already. ## Reply by Chad Sweet (ModalAI staff) · 2021-05-05 15:54:05 UTC Can you send the screenshot from the flashing process? Sometimes you need to run with 'sudo' or manually install via 'sudo fastboot' if you type 'sudo fastboot devices' what do you see? ## Reply by PawelJ · 2021-05-05 16:19:11 UTC Hi @Chad-Sweet , Thanks for replying so quickly, I really appreciate it as this has been blocking for my work. ![a031a1d1-a49b-40fb-b20b-b963bc7879a4-image.png](https://forum.modalai.com/assets/uploads/files/1620231484057-a031a1d1-a49b-40fb-b20b-b963bc7879a4-image.png) When I check for fastboot devices I see ``` (test) pjaworsk@ABR04:~/src/voxl/voxl-vision-px4$ sudo fastboot devices [sudo] password for pjaworsk: 534625ca fastboot ``` ## Reply by modaltb (ModalAI staff) · 2021-05-05 16:43:37 UTC In the installer folder that gets unzipped, there's a `system-image` directory, which has a `flash_build_apps.sh`script. This is getting stuck on: ``` while [ "$(fastboot devices)" == "" ] do echo "[INFO] Waiting for fastboot..." sleep 2 done ``` Can you please try to run the installer with sudo? e.g. `sudo ./install.sh` Something with permissions can get funky with fastboot and running with `sudo` seems to get through the issue normally. ## Reply by PawelJ · 2021-05-05 17:06:11 UTC Thanks, that got the install through. You can ignore the panicked email I sent out earlier :joy: I was worried I had bricked the device and would have to wait for a usb expansion board to ship. That fixed the issue with VIO local position ned and it is now visible in QGC again. I still seem to have the issue with my radio transmitter. QGC vocalizes the commands as usual, but I can't seem to arm the motors. When I go to the radio page of QGC it gives me this error![Screenshot from 2021-05-05 12-59-57.png](https://forum.modalai.com/assets/uploads/files/1620234238924-screenshot-from-2021-05-05-12-59-57-resized.png) It looks like this could have to do with the calibration [going off the docs](https://mavlink.io/en/messages/common.html#MAV_CMD_PREFLIGHT_CALIBRATION), but if I try to recalibrate the same error comes up. ## Reply by PawelJ · 2021-05-05 19:55:28 UTC It looks like I spoke too soon. The VIO data was sent after the initial reinstall. Once I tried compiling and building the `dev` branch of `voxl-vision-px4` (to use the polynomial path planning) it stopped sending to QGC again. I tried installing the master branch to fix it, but it still isn't sending. Following the same steps I listed above that I followed from [this forum post](https://forum.modalai.com/topic/140/vio-indoor-navigation/10?_=1620147287892), The `voxl-test-vision-lib` script functions as expected and `voxl-vision-px4` `voxl-camera-server` and `voxl-qvio-server` are running. ``` ~ # systemctl status voxl-vision-px4 ● voxl-vision-px4.service - voxl-vision-px4 Loaded: loaded (/usr/bin/voxl-vision-px4; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-05-05 19:45:52 UTC; 14s ago Process: 11248 ExecStartPre=/bin/sleep 2 (code=exited, status=0/SUCCESS) Main PID: 11259 (voxl-vision-px4) CGroup: /system.slice/voxl-vision-px4.service └─11259 /usr/bin/voxl-vision-px4 May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting fixed pose input May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting vio manager May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting apriltag manager May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting voa manager May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting offboard figure eight May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Init complete, entering main loop May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Connected to voxl-qvio-server May 05 19:45:55 apq8096 voxl-vision-px4[11259]: done updating transforms to use imu: imu1 May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Added new UDP connection to 192.168.86.176 May 05 19:45:55 apq8096 voxl-vision-px4[11259]: PX4 Connected over UART with sysid 1 ~ # systemctl status voxl-qvio-server ● voxl-qvio-server.service - voxl-qvio-server Loaded: loaded (/usr/bin/voxl-qvio-server; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-05-05 19:23:52 UTC; 22min ago Main PID: 6081 (voxl-qvio-serve) CGroup: /system.slice/voxl-qvio-server.service └─6081 /usr/bin/voxl-qvio-server May 05 19:23:52 apq8096 voxl-qvio-server[6081]: LNX_IA64 supported? 1 May 05 19:23:52 apq8096 voxl-qvio-server[6081]: WINDOWS supported? 0 May 05 19:23:52 apq8096 voxl-qvio-server[6081]: AR ERROR: arFileOpen(): Failed to open file: //vislam/Configu....xml May 05 19:23:52 apq8096 voxl-qvio-server[6081]: FASTCV: fcvAvailableHardware Linux May 05 19:23:52 apq8096 voxl-qvio-server[6081]: mempool cur block size 307200, new block size 307200 May 05 19:23:52 apq8096 voxl-qvio-server[6081]: Please ignore the error about Configuration.SF.xml above. ^^^ May 05 19:23:52 apq8096 voxl-qvio-server[6081]: It's an optional file, and should be a warning not an error May 05 19:23:52 apq8096 voxl-qvio-server[6081]: waiting for imu May 05 19:23:52 apq8096 voxl-qvio-server[6081]: waiting for cam May 05 19:44:32 apq8096 voxl-qvio-server[6081]: WARNING: output data thread fell behind Hint: Some lines were ellipsized, use -l to show in full. ~ # systemctl status voxl-camera-server ● voxl-camera-server.service - voxl-camera-server Loaded: loaded (/usr/bin/voxl-camera-server; disabled; vendor preset: enabled) Active: active (running) since Wed 2021-05-05 19:45:33 UTC; 56s ago Main PID: 10989 (voxl-camera-ser) CGroup: /system.slice/voxl-camera-server.service └─10989 /usr/bin/voxl-camera-server -c /etc/modalai/voxl-camera-server.conf May 05 19:45:34 apq8096 bash[10989]: AEAlgo : mvcpa May 05 19:45:34 apq8096 bash[10989]: mvcpa_filter_size : 2 May 05 19:45:34 apq8096 bash[10989]: mvcpa_exp_cost : 0.7500 May 05 19:45:34 apq8096 bash[10989]: mvcpa_gain_cost : 0.2500 May 05 19:45:34 apq8096 bash[10989]: mvcpa_histogram : false May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/hires_preview/ channel: 0 May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/stereo/ channel: 1 May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/tracking/ channel: 2 May 05 19:45:34 apq8096 bash[10989]: tracking May 05 19:45:34 apq8096 bash[10989]: Camera Width: 640, Height: 480, Format: 291 not supported! ``` However, my `voxl-imu-server` status is different, and I'm not sure if this is as it's expected to be. It also remains like this after I restart the service ``` ~ # systemctl status voxl-imu-server ● voxl-imu-server.service - voxl-imu-server Loaded: loaded (/usr/bin/voxl-imu-server; enabled; vendor preset: enabled) Active: activating (start-pre) since Wed 2021-05-05 19:49:03 UTC; 1s ago Process: 11969 ExecStart=/usr/bin/voxl-imu-server (code=exited, status=255) Main PID: 11969 (code=exited, status=255); : 11975 (sleep) CGroup: /system.slice/voxl-imu-server.service └─control └─11975 /bin/sleep 2 ``` I've included a list of my `voxl` and `modal` packages since the update ``` / # opkg list-installed | grep "voxl" libvoxl_io - 0.5.4 voxl-camera-server - 0.5.7 voxl-dfs-server - 0.0.7 voxl-docker-support - 1.1.1 voxl-gphoto2 - 0.0.5 voxl-hal3-tof-cam-ros - 0.0.5 voxl-imu-server - 0.7.8 voxl-modem - 0.11.0 voxl-mpa-tflite-server - 0.0.2 voxl-mpa-tools - 0.1.6 voxl-nodes - 0.1.3 voxl-qvio-server - 0.2.1 voxl-rtsp - 1.0.3 voxl-streamer - 0.2.1 voxl-suite - 0.3.4 voxl-utils - 0.6.0 voxl-vision-px4 - 0.8.1 voxl-vpn - 0.0.3 / # opkg list-installed | grep "modal" libmodal_json - 0.3.4 libmodal_pipe - 1.7.9 modalai-vl - 0.1.3 ``` Just for completeness, I've also included my `voxl-vision-px4.conf` ``` /** * VOXL Vision PX4 Configuration File * */ { "qgc_ip": "192.168.86.208", "en_localhost_mavlink_udp": true, "en_secondary_qgc": false, "secondary_qgc_ip": "192.168.1.214", "qgc_udp_port_number": 14550, "localhost_udp_port_number": 14551, "udp_mtu": 512, "en_vio": true, "en_voa": true, "en_send_vio_to_qgc": true, "en_send_voa_to_qgc": false, "en_set_clock_from_gps": true, "en_force_onboard_mav1_mode": true, "en_reset_px4_on_error": true, "qvio_auto_reset_quality": 0.00050000002374872565, "en_adsb": false, "adsb_uart_bus": 7, "adsb_uart_baudrate": 57600, "px4_uart_bus": 5, "px4_uart_baudrate": 921600, "offboard_mode": "figure_eight", "follow_tag_id": 0, "en_apriltag_fixed_frame": false, "fixed_frame_filter_len": 5, "en_transform_mavlink_pos_setpoints_from_fixed_frame": false } ``` ## Reply by PawelJ · 2021-05-05 20:19:11 UTC I've sent a few messages and listed a few problems above, so I just want to reiterate the current problem that's lead me here to stay on track. I'm trying to set a custom path and record the VIO position output (local_position_ned) from the named pipe while the path is being followed. My plan is to use the `dev` branch of `voxl-vision-px4` and incorporate something like [this example](https://gitlab.com/voxl-public/modal-pipe-architecture/libmodal_pipe/-/blob/master/examples/modal-hello-client.c) from `libmodal-pipe` to do the reading. This would be incorporated into the path script so it gets run whenever the offboard mode is following the path. I still have to look into the polynomial path planning code so can't speak on that yet. For the use case I imagine I would still have manual control from the transmitter so I can manual set the kill switch and take over control if anything happens. I also use this to enable offboard mode. Given the problems above, I can't use the transmitter to set the mode on the drone, so I can't test out any offboard flight. Since I'm also embedding the vio feedback recording in that path, I can't run that without switching to offboard mode. I imagine the issue of VIO not being sent to QGC is related to the transmitter communicating with QGC, but not with the drone (no state change or motor arming). ## Reply by Guest · 2021-05-06 18:39:25 UTC Hi Pawel, It's hard to tell exactly what's going on from these cutouts, would you be available for a short 30ish minute call where you can explain exactly what the issue is and I can help real time to figure out what's wrong, I think it'll be much quicker than going back and forth here. ## Reply by PawelJ · 2021-05-06 19:23:39 UTC Hi @Alex-Gardner , That would be great! We can do it over a video chat if that works. Feel free to send me a link at my email pawel.jaworski@appliedbrainresearch.com. Thank you ## Reply by PawelJ · 2021-05-13 20:42:13 UTC Just posting an update after some discuission with @Alex-Gardner. Thank you again, you were incredibly helpful in understanding the overall voxl comm pipeline. **Solution for Python I/O** The easiest method to do this was to run a modified vvpx4 that would record vio position based on a flag. Using the dev branch, I modified `process_new_vio_data()` in `vio_manager` [found here](https://gitlab.com/voxl-public/modal-pipe-architecture/voxl-vision-px4/-/blob/master/src/vio_manager.c#L85). I just added a snippet that would write `T_body_wrt_local.d` to a file in `/data/my_folder`. I have the flag set to start and stop logging when an offboard flight mode is started and stopped, by setting a flag in the `HOME` block of [the path](https://gitlab.com/voxl-public/modal-pipe-architecture/voxl-vision-px4/-/blob/master/src/offboard_figure_eight.c#L193). **Solution for QGroundControl and Transmitter Connection Problems** I had to reset the px4 parameters and recalibrate. This fixed my connectivity issues and I can now change modes and arm the motors through my transmitter. I can also see VIO data in QGC as well. The process I followed for this portion was based off the solution to [this forum post](https://forum.modalai.com/topic/105/voxl-m500-vision-problem/13) and the steps in the [docs here](https://docs.modalai.com/upload-px4-parameters/): - reset px4 parameters to firmare's default - load the [m500_4s_config.params](https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/blob/master/platforms/v1.10/m500_4s_config.params) - the instructions say to do this twice, however I think this is outdated as it didn't seem necessary for me, as the second time I went to upload the params I got a "no changes made" type notification - load the [mavlink_serial_ports.params](https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/blob/master/helpers/mavlink_serial_ports.params) helper config - run through the sensor calibration steps on QGC, I used a flat sheet of mdf on a large turntable to keep rotate the drone. Standing on its 4 legs and upside-down was fine, but for the other 4 orientations I would balance it on two legs. It would stay stable, but would be slightly off from a perfect right angle. From the PX4 docs it looks like this is fine for [accelerometer calibration since it uses a least squares fit](https://docs.px4.io/master/en/config/accelerometer.html). **Unresolved Problems** There is still something wrong here based on how the drone is flying, and I cannot tell whether it's related to the calibration or some missing parameters. It mostly flies fine in manual mode, maybe a little bit of drift, but it seems about the same as it was when I received the drone calibrated. In position control it starts to drift when in flight, and when stationary the VIO output drifts as well. Here is [a video I took](https://photos.app.goo.gl/bgwiXFXxSR1cG7sH9) of it "in the field." Appologies for the cellphone video, I had no way of screencapturing at the time. You can see the drone is stationary and it's still drifting. I don't think it has to do with trackable features, as this setup worked with when I first got the drone. I also took [a video of the flight](https://photos.app.goo.gl/GjxfRpgSbKHkBoKm7), going from manual, to position, to offboard (I say in the video when I change modes). You can see the drift here as I try to keep the drone centered in the net. The vio drift video with the drone stationary is with my modified vvpx4 where the vio position is saved when in offboard mode, so this should make no changes. However, I also tested with the master branch of vvpx4, which is the second video with the drone flying. Since it happens on the master branch, I think the problem is related to the QGC/PX4 setup and calibration. **Q1** One possible problem is that the [compass calibration](https://docs.px4.io/master/en/config/compass.html) says to stay away from metal, so possibly the 12" turntable was causing some interference, but I don't think it is likely because of the thick mdf on top of it. One point I am unsure of is whether the gps is required for this, as where I calibrate has no gps signal. Can you confirm whether gps is required for this step (I imagine not since you offer a gps free version)? **Q2** One other possible problem is that when I look at my px4 parameters, I do not see the `PWM_MIN` parameter listed in [the docs](https://docs.modalai.com/upload-px4-parameters/) when testing if the parameters were uploaded, which could be a sign of some missing px4 configs. From a fresh QGC/PX4 setup, is there something other than the `mavlink_serial_ports` and `m500_4s_config` that is required? ## Reply by PawelJ · 2021-05-18 14:57:20 UTC I reran the calibration outdoors with gps signal and far from any metal. I also raised the drone to be ~3-4' from the metal turntable below it. This improved the calibration quite a bit and the drone is not drifting as much as before. However, there still seems to be more drift in the VIO than there was when I first received the drone. I still don't have the `PWM_MIN` parameter that the docs say I should check for the assure the PX4 parameters loaded properly. I see that this parameter is in the [custom2.params](https://gitlab.com/voxl-public/flight-core-px4/px4-parameters/-/blob/master/platforms/v1.10/custom2.params#L75) file, but the note in the document says it turns off the gps. Can you please explain the steps you go through during your calibration process? Should I also be loading this `custom2.params` file? If so, does that mean that gps is not used when in VIO (no sensor fusion etc)?