Posts made by svempati

@Zachary-Lowell-0 Thanks for sharing the video! I looked at it, and I pretty much did the same steps you did. It might be worth mentioning that I had to modify the Dockerfile because the one in the documentation was throwing a version mismatch error when installing the onnx part.

This was the original docker command

RUN pip3 install ultralytics tensorflow onnx "onnx2tf>1.17.5,<=1.22.3" tflite_support onnxruntime onnxslim "onnx_graphsurgeon>=0.3.26" "sng4onnx>=1.0.1" tf_keras

I modified it to this

RUN pip3 install ultralytics tensorflow "onnx2tf>1.17.5,<=1.22.3" tflite_support onnxruntime onnxslim "onnx_graphsurgeon>=0.3.26" "sng4onnx>=1.0.1" tf_keras
RUN pip3 install onnx==1.20.1

I don't think this should cause any issues, but could you confirm?

@Zachary-Lowell-0 I would first like to diagnose what is causing the yolov8 model to not work on the voxl 2 for me. Will it only work when you train the model/ export it to tflite on a TPU? I am getting the issue even if I train the yolov8 model on an open source dataset/ use the pretrained yolov8n.pt model downloaded from ultralytics. I want to make sure I can train a yolov8 model on an open source dataset from scratch works on the voxl, so that I can move on to using my custom dataset.

In case there is no other workaround then I should be able to send you the dataset I am using.

Thanks!

@Zachary-Lowell-0 Just wanted to follow up to see if you were able to replicate this issue?

@Zachary-Lowell-0 I wanted to follow up with you again on this, and the issue seems to be the model conversion process from pytorch to tflite. To confirm this, I tried it with the default yolov8n.pt downloaded from ultralytics by entering this command from the gitlab repository:

python export.py yolov8n.pt

So that I create a new yolov8n_float16.tflite file. However, running this file on voxl-tflite-server shows this output before displaying Error in TensorData<float>: should not reach here:

WARNING: Unknown model type provided! Defaulting post-process to object detection.
INFO: Created TensorFlow Lite delegate for GPU.
INFO: Initialized OpenCL-based API.
INFO: Created 1 GPU delegate kernels.
Successfully built interpreter

------VOXL TFLite Server------

 4 5 6
 4 5 6
Connected to camera server

I even tried running export.py on the voxl emulator to account for any differences in the cpu architecture between my computer and the VOXL (between X86 and ARM) but I still get the same error. Do you think there is anything I would be missing? Thank you!

@Zachary-Lowell-0 Got it, I will try that out and will let you know if I have any more questions. Thanks for your help!

I see, so my model is not supported by the voxl-tflite-server since it is float16 and the tflite server only supports 32 bit precision if we want to use floating point values. Am I understanding that correctly or am I missing something? Cause the default YOLOv5 model that is included in the VOXL 2 model (yolov5_float16_quant.tflite) is also of float 16 precision so I wonder how the functions in tensor_data.h handle that.

One question, what command did you use to view these error logs from voxl-tflite server?

Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here
Error in TensorData<float>: should not reach here

Hi @Zachary-Lowell-0, Yes I am confirming that I followed the instructions in that gitlab repository.
Here is the tflite file and labels file: https://drive.google.com/drive/folders/1kyjanabVSP_pH_jsQyjQG9z6hFYZ_iij?usp=drive_link

Hello,

I am trying to run a custom YOLOv8 model on the voxl-tflite-server. The model detects ships and the yolov8_labels.txt file only contains one ship class. However, when I run the tflite server and view it on voxl-portal I can see the video feed, but cannot see any bounding box detections even when the target is in the camera frame.
I tried another variation for the labels file by having the class index and label name like this: 0 ship, but that doesn't work either.
I also ran voxl-inspect-detections but it doesn't show any detections there.

When I tested the default yolov5 and yolov8 models on voxl-tflite-server, it displays the bounding boxes and shows the list of detections in voxl-inspect-detections just fine.

If it helps, I used this command to convert the YOLOv8 model to the tflite format:

yolo export model=best.pt format=tflite

I use the quantized 16 bit tflite model named yolov8_best_float16.tflite.

This is how I set up the config file /etc/modalai/voxl-tflite-server.conf:

{
"skip_n_frames":	0,
"model":	"/usr/bin/dnn/yolov8_best_float16.tflite",
"input_pipe":	"/run/mpa/front_small_color/",
"delegate":	"gpu",
"requires_labels":	true,
"labels":	"/usr/bin/dnn/yolov8_labels.txt",
"allow_multiple":	false,
"output_pipe_prefix":	"yolov8"
}

Is there anything I missed that is leading to no detections on the voxl-tflite-server?

I would appreciate any help!

@tom No I don't have a wifi dongle connected.

@tom I ran ifconfig on voxl2, but I haven't found anything on wlan0. This is the output I get after running that command:

bond0: flags=5123<UP,BROADCAST,MASTER,MULTICAST>  mtu 1500
        ether fe:ca:d2:74:91:69  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

dummy0: flags=195<UP,BROADCAST,RUNNING,NOARP>  mtu 1500
        inet6 fe80::e2a:ce35:6c11:d803  prefixlen 64  scopeid 0x20<link>
        ether de:92:20:5e:07:c9  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 25  bytes 7462 (7.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 368130  bytes 35420354 (35.4 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 368130  bytes 35420354 (35.4 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

would any of these help with finding the voxl's ip address?

Hello, I am trying to read mavlink messages like the pressure and altitude from the VOXL 2 sensors using pymavlink. I wanted to make sure that the udpin to the mavlink connection is correct?

I am having trouble connecting to the mavlink server. I set the port number to 14557 based on the first diagram in this page: (https://docs.modalai.com/mavlink/).
When I use localhost as the port, . When I use localhost, I get an error that says OSError: [Errno 98] Address already in use, and when I use the IP address 192.168.8.10 it gives an error that says OSError: [Errno 99] Cannot assign requested address.

If that is not the correct IP address, how can I find the IP address for my specific VOXL? It is worth noting that my VOXL isn't connected to the internet, but is physically connected to the flight controller by wire.

Thanks for your help!

# MAVLINK Common Message Set: https://mavlink.io/en/messages/common.html
from pymavlink import mavutil

the_connection = mavutil.mavlink_connection('udpin:192.168.8.10:14557')

the_connection.wait_heartbeat()
print("Heartbeat from system (system %u component %u)" % (the_connection.target_system, the_connection.target_component))

while True:
    try:
        # local xyz position
        msg_pos = the_connection.recv_match(type='LOCAL_POSITION_NED', blocking=True)
        print(msg_pos.x, msg_pos.y, msg_pos.z)

        # altitude
        msg_altitude = the_connection.recv_match(type='ALTITUDE', blocking=True)
        print(msg_altitude.altitude_local, msg_altitude.altitude_relative)

        # pressure
        msg_pressure = the_connection.recv_match(type='SCALED_PRESSURE', blocking=True)
        print(msg_pressure.press_abs)

    except KeyboardInterrupt:
        print('Terminating mavlink...')
        break

@Alex-Kushleyev I haven't yet but will try doing that. I just wasn't sure how to configure the camera for VIO. Thank you!

I am trying to implement Visual Inertial Odometry using the QVIO server to find the local XYZ positions, but the documentation only mentions it working with the tracking camera (Camera and IMU Calibration section in https://docs.modalai.com/voxl-qvio-server-0_9/#resetting-vio).

Would I be able to use the OV7982 image sensor on VOXL 2 so that I get a similar output as the one below when I run voxl-inspect-qvio? If that would work, are there additional configurations or parameters I need to change before running VIO?

yocto:/$ voxl-inspect-qvio

    T_imu_wrt_vio (m)   |Roll Pitch Yaw (deg)| state| error_code
   -4.96    0.94   -0.00|  17.9  -52.3    9.3| OKAY | 

@Alex-Kushleyev Got it. Thank you your help and for taking the time to answer my questions!

Sashank

@Alex-Kushleyev By storing to mpa I mean creating a new service under voxl-inspect-services and storing px4 messages under that new service.

I understood that px4-listener could be directly executed in python3, but I was assuming that listening to data from one of the voxl-services would be the a faster or more efficient way to get data on the VOXL board.

Sashank

@Alex-Kushleyev So what I wanted to ask was whether you could pipe px4 messages in general into mpa. The idea is I would like to do something like this:

px4-listener sensor_baro -> some measurements -> Get pressure measurement -> store to mpa -> read mpa pipe from python

Would it be possible to create a pipeline that looks something like this?

@Alex-Kushleyev Thank you for your response, I will try that out. I have a follow up question, is there a way to make a pipe from the px4 listener that will be running in the background?

I see that in the list of voxl-inspect-services, there isn't a service to read barometer data. It looks like the only ways to read the data from the onboard barometer on the VOXL 2 is to either use the PX4 Flight controller or the I2C address given in the page on onboard sensors (https://docs.modalai.com/voxl2-onboard-sensors/). I ultimately want to create a python script that listens to the barometer data.

How do I create a new pipe using MPA that publishes barometer data?