I’m a total loss with making my Tfiite models work on the VOXL2.

I’ve followed every thread.

I’ve successfully trained models using the recommended Tensorflow, i’ve quantized it down to 16 float.

i can successfully run these on my linux box, my macbook pro, a raspberrypi and the nVidia Jetson Nano.

But when uploaded to VOXL2 i do get video but absolutely no detection what so ever.

Also per https://docs.modalai.com/voxl-tflite-server/ i followed the instructions for post training quantization on
my frozen graph and on my saved models.

I used Netron to find the input/output parameters.

I also successfully converted a YOLOv8 model into a tFlite model and ran object detection perfectly on all the different platforms except for the VOXL2.

Here's a link to one of my colab notebooks i've used to train an object detection model.
https://colab.research.google.com/drive/1QdgpSl63OSQdLTnFwOyP8dxLQ7W0HtmW?usp=sharing

@Moderator Ah ok. these are working!

just for everybody else's edification
instead of just typing "hires" you need to type one of the following listed with
voxl-list-pipes

voxl2:/$ voxl-list-pipes
hires_color
hires_grey
hires_large_color
hires_large_encoded
hires_large_grey
hires_small_color
hires_small_encoded
hires_small_grey
tflite
tflite_data

@Denver-Bennett said in How do I connect MAVROS to PX4 with UDP ports?:

voxl-vision-hub.conf

did you ever get this resolved? having same issue.
S

• /uas_20/mission_control_server/mission_filepath: /home/root/catkin...
• /uas_20/mission_control_server/use_navsatfix_launch: False

NODES
/uas_20/
mavros (mavros/mavros_node)
mission_control_server (demins_mavros/mission_control_server.py)

ROS_MASTER_URI=http://10.10.0.50:11311/

process[uas_20/mavros-1]: started with pid [2835]
process[uas_20/mission_control_server-2]: started with pid [2836]
[INFO] [1767215095.413547]: Waiting for MAVROS services...
[INFO] [1767215095.511182]: Waiting for mavros/mission/push service...
[ WARN] [1767215095.880973783]: DS: plugin not configured!
[INFO] [1767215097.183459]: Waiting for mavros/mission/pull service...
[INFO] [1767215097.254429]: Waiting for mavros/mission/clear service...
[INFO] [1767215097.313131]: Waiting for mavros/mission/set_current service...
[INFO] [1767215097.362325]: Waiting for mavros/set_mode service...
[INFO] [1767215097.412324]: Waiting for mavros/cmd/arming service...
[INFO] [1767215097.469221]: Waiting for mavros/param/set service...
[INFO] [1767215097.520002]: All MAVROS services are available!
[INFO] [1767215097.541382]: Clearing all waypoints from FCU...
[ WARN] [1767215098.608210241]: WP: timeout, retries left 2
[ WARN] [1767215099.609752740]: WP: timeout, retries left 1
[ WARN] [1767215100.610535292]: WP: timeout, retries left 0
[ERROR] [1767215101.611777063]: WP: timed out.
[ERROR] [1767215101.616417]: Failed to clear waypoints
[INFO] [1767215101.670286]: Waypoint Mission Manager Initialized.
[INFO] [1767215101.864318]: Mission Control Server Initialized.

i'm at a loss here. tried everything with ports etc

here's my launch script
roslaunch projectx voxl_end.launch mav_sys_id:=20 fcu_url:="udp://127.0.0.1:14540@127.0.0.1:14550"

@tom thanks tom!

with voxl-configure-sku -w
I assumed the option to select is
6) px4-autonomy-dev-kit

i then ran voxl-configure-mpa
saw a few errors but the qvio is working now with no errors.

Sabri

@tom i ran
journalctl -u voxl-qvio-server
and saw this message as the extrinsics.conf file doesnot exist

failed to find extrinsics from imu_px4 to tracking in /etc/modalai/extrinsics.conf

@tom i did that and tried restarting the voxl-qvio-server but it wont start.

i've followed all instructions from calibrating the tracking camera
https://docs.modalai.com/calibrate-cameras

to
https://docs.modalai.com/voxl-m500-user-guide-using-vio

i've configured every setting with voxl-configure-qvio and still get the following error

voxl2:/$ voxl-inspect-qvio
waiting for server

dt(ms) | T_imu_wrt_vio (m) |Roll Pitch Yaw (deg)|features|quality| state| error_codes |
30.9 | 0.00 0.00 0.00| 0.0 -0.0 0.0| 0 | -1% | FAIL | IMU_MISSING STALLED

any ideas?

@bgaudel Did you ever get this resolved. I'm experiencing the same thing with IMU_MISSING STALLED

@thomas I already wrote a c helper function that uses libmodal-pipe to pipe tflite_data that my python script reads in realtime. it works well
per your message " small C++ helper program to take in images from the camera". how would i modify my c helper to use libmodal-pipe to stream data from one of the cameras?

@thomas i'm not using voxl-tflite-server. On my voxl2 i installed the latest version of YOLOv8 from Ultralytics https://docs.ultralytics.com/ and it works well when i run inference on local images. i wanted to test just using a local python script that streams video frames from the hires_small_color camera and runs inference.

@thomas Hey thomas, we've got the handshaking between your C helper program and our python script working well. i successfully loaded Yolov8 on the voxl and it works really well on images. Was wondering if we could open the camera pipe the same as the tflite_data pipe to run inference on the frames in realtime. i noticed in your /data/web_root/index.html that you are using websockets for serving the videos in the browsers. is this the better route?