voxl-mpa-to-ros publish only some tflite data

[planch98]

Hi everyone,
I'm working with tflite to do object detection on VOXL2, sdk 0.9.5. The output is then sent to ros by voxl-mpa-to-ros that publishes tflite_data topic. I'm facing the following issue: only a part of the ai_detection messages within the same frame are sent. I've verified this by running rostopic echo /tflite_data and comparing it with the tflite image in which I print the frame ID. This way I can see that NOT all the detections present in a single frame in voxl-portal are actually sent via voxl-mpa-to-ros.

Why am I not able to get all the detections present in one frame from voxl-mpa-to-ros? Has this ever happened to anyone? How can I fix this?

[Moderator]

Which model are you using? they all publish data slightly differently

For YOLO, you can see the code where the ai_detection is published here: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/src/main.cpp#L243

The mpa-to-ros code where the detection is received is here: https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-to-ros/-/blob/master/catkin_ws/src/src/interfaces/ai_detection_interface.cpp?ref_type=heads#L98

[planch98]

Thanks for your answer, I'm using a custom-trained yolov5 model with 2 classes

[Zachary Lowell 0]

@Moderator said in voxl-mpa-to-ros publish only some tflite data:

detection

Hi @planch98 I will be testing this on the base yolo5 weights we have running in tflite. I am under the impression that maybe the publisher is to constricted and only letting a specific amount of detections through the pipe. Will keep you posted. Also @planch98 what is the max number of detections you are seeing in your image and what is the max number of outputs you are seeing in the voxl-mpa-to-ros echo message?

[planch98]

Hi @Zachary-Lowell-0, I've had the same thought, so I lowered the frame rate of the hires camera to 10 FPS. This proved to solve some lag issues but the publishing issue remains.

I've performed another test: since I thought that the detections were too many, I've edited voxl-tflite-server to exclude one of the 2 classes (the most frequent one) from the detection pipe, publishing only the less frequent one (roughly 3-5 detections per frame). To do this, I've simply put an if to filter the detections by looking at the class name at line 243 (https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/src/main.cpp?ref_type=heads). Unfortunately, the issue still persists.

I will run the tests that you asked and I'll let you know asap. Thanks for your support.

[planch98]

Hi @Zachary-Lowell-0, sorry for the late reply.
I've run the test that you asked with the new code that you wrote (v0.3.7). The results were pretty random, we go from a lower bound of 3 detections sent through voxl-mpa-to-ros over 10 detections present in tflite, to an upper bound of 8 over 11.
This happens also for a small number of detections i.e. 2/4 as lower bound to 3/3 as an upper bound.

Are your tests aligned with mine? I'm still using a custom yolo v5 for 2 class detections.