Implementing a Custom YOLOv5 Model on APQ8096 VOXL-TFLite-Server
-
Hello!
I'm currently undertaking a project involving a custom retrained YOLOv5s model. The model was trained using PyTorch, resulting in .pt format weights. I've tested these weights successfully using YOLOv5 ROS detection with real-time drone imagery from the
\trackingtopic published by voxl-mpa-to-ros.However, I noticed a significant latency between the output of YOLOv5 ROS detection and the real-time drone position. To minimize this, I'm exploring the use of voxl-tflite-server for onboard implementation.
Your documentation specifies that the server on VOXL1 can only run TFLite models using the v2.2.3 supported opsets. My attempts to implement a .tflite model exported using a newer TensorFlow version have so far been unsuccessful. I used YOLOv5's export script to export the .pt model to .tflite and attempted to rebuild and deploy the voxl-tflite server on the drone, but there was no response on the VOXL portal.
In another attempt, I converted a frozen graph (.pb model) to a .tflite model using your provided Python script. Despite using TF v2.2.3 for compatibility, the conversion process was hindered by an attribute ('explicit_paddings') in the MaxPool node that the older TensorFlow version does not recognize. This attribute appears to be a feature of a newer TensorFlow version.
In light of these challenges, I am wondering whether I should retrain the model using TensorFlow v2.2.3. Any advice or guidance on how best to proceed would be greatly appreciated.
Moreover, regarding the integration of a functional .tflite model with the voxl-tflite-server. The documentation mentions the need to write and integrate a custom post-processing function into the InferenceHelper. Given that my custom model is essentially a YOLOv5 model with different weights, can I assume that a post-processing function similar to the YOLOv5 function integrated in the package would suffice? Also, are there additional modifications necessary for full integration and streaming on VOXL portal?
Finally, to provide context for this project, my ultimate goal is to publish the detection output as a ROS message and adjust the drone's position based on this detection somehow. It’s worth mentioning the ultimate goal out of this to make the whole picture clear in case there is another way around which could be less complicated
As I'm relatively new to machine learning and TensorFlow Lite, your guidance and correction on any misconceptions I might have would be highly beneficial. Thank you in advance for your assistance and support.
-
Hey Everyone!
I've made some progress on this and I thought it'd be a good idea to leave this reply here on this question for future similar inquiries. I've compiled everything into a guide for running a custom classifier on voxl-tflite-server - from collecting data to training, testing, and deploying it on a drone.
Here's the link to the GitHub repo: CustomYOLOv5-NIST-Buckets-VOXL-tflite. Check it out if you're working on something similar - it might save you some time.
If you find it useful, give it a star to help others discover it.
Thanks! -
@aashry This looks very helpful! A team member responsible for tuning object detection spent hours trying to abide by the ModalAI comment in their scripts to use TFLite version 2.2.3 or OLDER. The comment that modalAI provides in that script (but not in any documentation) seems so strict. Just getting that version of TFLite installed somewhere was a headache, and then the model he exported with it didn't work.
What worked for him was to use the export script provided by YOLOv5, as you say. That exported model just worked fine for him, and it was very straightforward. It would be nice if modalAI would update their scripts to not make people jump through TFLite 2.2.3 (or earlier) hoops when in the end they'll probably have more success using the native export scripts from YOLO.
-
@tpavlic Thank you, glad it helped!