ModalAI UAS Technical Questions
-
My name is Mario Curtis and I’m currently helping a customer with on-board AI processing on small UAS platforms. We purchased a Sentinel and a Mini for development and prototyping and I have a couple of questions which I hope you can answer.
-
Is there only going to be support for yolov5? Our development, annotation and modeling pipelines are all setup for yolov8 as most systems are. Is there a conversion mechanism preferred for the model translation from v8 to v5 coco (TFLITE)?
-
We currently use Roboflow to annotate and train the models. There is a option to download a yolov5 BBO version or a Yolov5 Pytorch. Is it possible to use either of those datasets and use the native yolov5 convert to tflite to run our models. Their docs are not super detailed on creating models so any advice would be of great help.
-
Recent SDK 1.1.2 (Sentinel) update for voxl-configure-tflite does not map to the correct hires. There is a manual change which must be done on voxl-tflite.conf.
-
We were introduced to ModalAI from AFWERKS we want to know if there will be continuous integration with QGCGOV.
-
voxl-inspect-services shows a heavy use with camera server (70 to 90 CPU percent). Is this a bug or is there something we should do to optimize since we are using object detection with the small hires color camera?
-
Is the ModalAI ESC board the only thing compatible with the controller or is there any possibility of using a third party ESC.
-
Is there any documentation for SLAM on Sentinel with the SDK or is it only native to ROS?
Any information would be of great help, thank you
-
-
Is there only going to be support for yolov5. Our development, annotation and modeling pipelines are all setup for yolov8. Is there a conversion mechanism preferred for the model translation from v8 to v5 coco?
What we have in our VOXL SDK is the voxl-tflite-server which is a reference for how to integrate TensorFlow Lite with the rest of our VOXL SDK. It is common for our customers to modify this if they want to use a different model. We are always ready to collaborate and it is great when our customers submit pull requests so we can incorporate other capabilities that might be useful.
We currently use Roboflow to annotate and train the models. There is an option to download a yolov5 BBO version or a Yolov5 Pytorch. Is it possible to use either of those datasets and use the native yolov5 convert to tflite to run our models. Their docs are not super detailed on creating models so any advice would be of great help.
We’re not very familiar with Roboflow. Have you seen this thread regarding Google Colab? That might be relevant for what you are trying to do.
Recent SDK 1.1.2 (Sentinel) update for voxl-configure-tflite does not map to the correct hires. There is a manual change which must be done on voxl-tflite.conf.
We can investigate, if you have a diff of the change we can incorporate in future releases more quickly.
will there be continuous integration with QGCGOV.
We have tried with QGCGOV, and have plans to support longer term, but have not put a large effort into it. We certainly test against stock QGC daily, so do not expect a significant effort to keep compatibility.
voxl-services shows a heavy use with camera server (70 to 90 percent). Is this a bug or is there something we should do to optimize since we are using onboard object detection with the small hires color camera?
That seems high, what else is running? Are you encoding multiple streams or something? Look at voxl-inspect-cpu, that might give you a better indication of overall performance.
Is the ModalAI ESC board the only thing compatible with the controller or is there any possibility of using a third party ESC.
See here for use with PWM ESCs
Is there any documentation for SLAM on Sentinel with the SDK or is it only native to ROS?
Start with voxl-mapper. It is possible use with stereo sensors, like on Sentinel, but those are more noisy that TOF (which is better for indoors).
-
For the LTE Modem on the Sentinel will the Modem cover European bands for operation? (normally there is US and European Variant).
Do you have any LEPTON Thermal sensor ready for the Sentinel without disabling or modify the drone? Will a USB connection to the drone simply be viewed as an additional camera (FLIR C5) in the OS or will we have to solder the connections on the flight controller?
