Can anyone recommend a Tflite Colab Notebook for VOXL2 Training
-
Of these 4 models which is the custom trained YOLO model you mentioned above? I can't even get the
edgetpu.tflitemodel to load so I need a little bit more information on how each of these files was generated.Thomas
@thomas here's a yolov5 trained model.
https://drive.google.com/file/d/1wRbIXdylgx-EOGDLnuWnsytGd2DcTNeY/view?usp=drive_linkI used this instruction set to train a yolov5 model which works well on my mac, rPI4, linux box and nvidia jetson.
https://docs.ultralytics.com/yolov5/tutorials/train_custom_data/#23-organize-directories -
@thomas here's a yolov5 trained model.
https://drive.google.com/file/d/1wRbIXdylgx-EOGDLnuWnsytGd2DcTNeY/view?usp=drive_linkI used this instruction set to train a yolov5 model which works well on my mac, rPI4, linux box and nvidia jetson.
https://docs.ultralytics.com/yolov5/tutorials/train_custom_data/#23-organize-directories@thomas here's the colab notebook i used to create the tflites and quantized tflites
https://colab.research.google.com/drive/1QdgpSl63OSQdLTnFwOyP8dxLQ7W0HtmW?usp=sharing -
@thomas here's the colab notebook i used to create the tflites and quantized tflites
https://colab.research.google.com/drive/1QdgpSl63OSQdLTnFwOyP8dxLQ7W0HtmW?usp=sharingI found the bug in our code that's causing the issue. You can see on this line that a for YOLO models a former employee hardcoded the number of classes to be 80 leading to a segfault when we only have a single class like in your case. Here's what your model looks like on our server with the fix:
What I'm going to do is write a patch for
voxl-tflite-serverthat more intelligently reads the number from the labels file. This will be available in our SDK Nightlies and so you'll be able to use the fix tomorrow morning. If you'd like the fix sooner, what I can do is package the new voxl-tflite-server into a .deb so you can deploy it manually to your VOXL. I can help you out with this process if you're not familiar.Thanks,
Thomas Patton
thomas.patton@modalai.com -
I found the bug in our code that's causing the issue. You can see on this line that a for YOLO models a former employee hardcoded the number of classes to be 80 leading to a segfault when we only have a single class like in your case. Here's what your model looks like on our server with the fix:
What I'm going to do is write a patch for
voxl-tflite-serverthat more intelligently reads the number from the labels file. This will be available in our SDK Nightlies and so you'll be able to use the fix tomorrow morning. If you'd like the fix sooner, what I can do is package the new voxl-tflite-server into a .deb so you can deploy it manually to your VOXL. I can help you out with this process if you're not familiar.Thanks,
Thomas Patton
thomas.patton@modalai.com@thomas
Fix TFLite Num Classes Bug (!22) · Merge requests · voxl / VOXL SDK / Services / voxl-tflite-server · GitLab
Tensorflow-Lite (TFlite) server that outputs Tflite processed images
GitLab (gitlab.com)
Here's the merge request with the fix, this will be published both in tonight's nightly and new SDK releases.
Thomas
-
I found the bug in our code that's causing the issue. You can see on this line that a for YOLO models a former employee hardcoded the number of classes to be 80 leading to a segfault when we only have a single class like in your case. Here's what your model looks like on our server with the fix:
What I'm going to do is write a patch for
voxl-tflite-serverthat more intelligently reads the number from the labels file. This will be available in our SDK Nightlies and so you'll be able to use the fix tomorrow morning. If you'd like the fix sooner, what I can do is package the new voxl-tflite-server into a .deb so you can deploy it manually to your VOXL. I can help you out with this process if you're not familiar.Thanks,
Thomas Patton
thomas.patton@modalai.com@thomas you rock! awesome! thanks for troubleshooting.
-
@thomas
Fix TFLite Num Classes Bug (!22) · Merge requests · voxl / VOXL SDK / Services / voxl-tflite-server · GitLab
Tensorflow-Lite (TFlite) server that outputs Tflite processed images
GitLab (gitlab.com)
Here's the merge request with the fix, this will be published both in tonight's nightly and new SDK releases.
Thomas
@thomas thanks thomas. will download tomorrow and test either in evening or next day. Really appreciate your help with this.
Great job! Sabri -
@thomas thanks thomas. will download tomorrow and test either in evening or next day. Really appreciate your help with this.
Great job! SabriYeah of course, glad I could be of assistance! Be sure to let me know if you need any more help with this.
Two things I was thinking about on my drive home that you should know - First, when you're on that page I linked above for the software nightlies make sure you sort the files by date descending so that you get the most recent one. I always make the mistake of getting the wrong one. Secondly, when you flash that nightly SDK it will overwrite anything not in the
/data/directory and so you will need to put your model file and labels txt back into/usr/bin/dnn/.Thomas
-
Yeah of course, glad I could be of assistance! Be sure to let me know if you need any more help with this.
Two things I was thinking about on my drive home that you should know - First, when you're on that page I linked above for the software nightlies make sure you sort the files by date descending so that you get the most recent one. I always make the mistake of getting the wrong one. Secondly, when you flash that nightly SDK it will overwrite anything not in the
/data/directory and so you will need to put your model file and labels txt back into/usr/bin/dnn/.Thomas
@thomas Morning Thomas, just getting back to this. I couldnt find any SDK that had a 2024 stamp. Was this uploaded somewhere else?
Sabri -
@thomas Morning Thomas, just getting back to this. I couldnt find any SDK that had a 2024 stamp. Was this uploaded somewhere else?
SabriHere's the one from last night:
https://console.cloud.google.com/storage/browser/_details/platform-nightlies/voxl2/voxl2_SDK_nightly_20240117.tar.gz?pageState=("StorageObjectListTable":("s":[("i":"objectListDisplayFields%2FtimeLastModified","s":"1"),("i":"displayName","s":"0")]))&organizationId=517175400245&project=modalai-core-services
You just have to make sure when you're on this page that you sort date modified by descending.
Thanks,
Thomas -
Here's the one from last night:
https://console.cloud.google.com/storage/browser/_details/platform-nightlies/voxl2/voxl2_SDK_nightly_20240117.tar.gz?pageState=("StorageObjectListTable":("s":[("i":"objectListDisplayFields%2FtimeLastModified","s":"1"),("i":"displayName","s":"0")]))&organizationId=517175400245&project=modalai-core-services
You just have to make sure when you're on this page that you sort date modified by descending.
Thanks,
Thomas@thomas
installed voxl2_SDK_nightly_20240117.tar.gz
per https://docs.modalai.com/flash-system-image/still not getting any detection even with your default yolov5. ive tried with hires large color, small color and grey.
-
@thomas
installed voxl2_SDK_nightly_20240117.tar.gz
per https://docs.modalai.com/flash-system-image/still not getting any detection even with your default yolov5. ive tried with hires large color, small color and grey.
-
@thomas @sansoy
So I was able to get our YOLO model running immediately, am about to try yours next. All I did was:
- Flash the nightly SDK from last night using the steps you linked above
- Edited
/etc/modalai/voxl-tflite-server.confso that the model was/usr/bin/dnn/yolov5_float16_quant.tfliteand the input pipe was/run/mpa/hires_small_color.
You should definitely try to reproduce this result as if you can't it might indicate a deeper problem.
Then to use your custom model:
adb shelland thencd /usr/bin/dnnand thencp yolov5_labels.txt yolov5_labels.txt.bak && cp yolov5_float16_quant.tflite yolov5_float16_quant.tflite.bakto back up the existing model and label files.vi yolov5_labels.txtand change to only havear15as a class- From outside ADB,
adb push yolo5_fp16_quant.tflite /usr/bin/dnn/yolov5_float16_quant.tflite voxl-tflite-server
Thomas
thomas.patton@modalai.com -
@thomas @sansoy
So I was able to get our YOLO model running immediately, am about to try yours next. All I did was:
- Flash the nightly SDK from last night using the steps you linked above
- Edited
/etc/modalai/voxl-tflite-server.confso that the model was/usr/bin/dnn/yolov5_float16_quant.tfliteand the input pipe was/run/mpa/hires_small_color.
You should definitely try to reproduce this result as if you can't it might indicate a deeper problem.
Then to use your custom model:
adb shelland thencd /usr/bin/dnnand thencp yolov5_labels.txt yolov5_labels.txt.bak && cp yolov5_float16_quant.tflite yolov5_float16_quant.tflite.bakto back up the existing model and label files.vi yolov5_labels.txtand change to only havear15as a class- From outside ADB,
adb push yolo5_fp16_quant.tflite /usr/bin/dnn/yolov5_float16_quant.tflite voxl-tflite-server
Thomas
thomas.patton@modalai.com@thomas WORKED! i did most of what you stated except for renaming my file to your filename. !
in a future release would be able to use the original filename vs renaming it to something that is hardcoded?
-
@thomas WORKED! i did most of what you stated except for renaming my file to your filename. !
in a future release would be able to use the original filename vs renaming it to something that is hardcoded?
Glad you got it working!
It's important to note that
voxl-tflite-serverisn't intended to be a high-grade tool for running any ML model with great configurations and optimizations; it's just an example of how you can use the VOXL ecosystem to run models. As much as I would enjoy building such a tool and implementing your request (and the others I get), there just isn't enough of a business demand for it right now.That being said, the code is open source and right here is the line that does the string comparison. If you want to put out a merge request to add a new field for
model_typeto the/etc/modalai/voxl-tflite-server.confJSON and have it select an inference method based on that, you can tag me and I'll review it.Thanks,
Thomas Patton
thomas.patton@modalai.com -
Glad you got it working!
It's important to note that
voxl-tflite-serverisn't intended to be a high-grade tool for running any ML model with great configurations and optimizations; it's just an example of how you can use the VOXL ecosystem to run models. As much as I would enjoy building such a tool and implementing your request (and the others I get), there just isn't enough of a business demand for it right now.That being said, the code is open source and right here is the line that does the string comparison. If you want to put out a merge request to add a new field for
model_typeto the/etc/modalai/voxl-tflite-server.confJSON and have it select an inference method based on that, you can tag me and I'll review it.Thanks,
Thomas Patton
thomas.patton@modalai.com@thomas said in Can anyone recommend a Tflite Colab Notebook for VOXL2 Training:
It's important to note that voxl-tflite-server isn't intended to be a high-grade tool for running any ML model with great configurations and optimizations; it's just an example of how you can use the VOXL ecosystem to run models. As much as I would enjoy building such a tool and implementing your request (and the others I get), there just isn't enough of a business demand for it right now.
Thanks for the info. how can i get access to the box coordinates in the image?
-
@thomas said in Can anyone recommend a Tflite Colab Notebook for VOXL2 Training:
It's important to note that voxl-tflite-server isn't intended to be a high-grade tool for running any ML model with great configurations and optimizations; it's just an example of how you can use the VOXL ecosystem to run models. As much as I would enjoy building such a tool and implementing your request (and the others I get), there just isn't enough of a business demand for it right now.
Thanks for the info. how can i get access to the box coordinates in the image?
Check out https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/include/ai_detection.h?ref_type=heads , this defines the detection output. This is streamed out to a pipe in
/run/mpaThomas
-
Check out https://gitlab.com/voxl-public/voxl-sdk/services/voxl-tflite-server/-/blob/master/include/ai_detection.h?ref_type=heads , this defines the detection output. This is streamed out to a pipe in
/run/mpaThomas
@thomas @sansoy Has anyone tried integrating a yolov8n model, I have a tflite fp16 quantized model but this line in the modal ai documentation says that any other model architecture apart from the ones mentioned here cannot be used,
file:///home/swarm_researchers/Pictures/Screenshots/Screenshot%20from%202024-01-27%2021-34-41.png -
@thomas @sansoy Has anyone tried integrating a yolov8n model, I have a tflite fp16 quantized model but this line in the modal ai documentation says that any other model architecture apart from the ones mentioned here cannot be used,
file:///home/swarm_researchers/Pictures/Screenshots/Screenshot%20from%202024-01-27%2021-34-41.png@sansoy has been able to get custom models working, but you'll have to follow the steps we took in the posts above. Let me know how I can best help!
Thomas
-
@sansoy has been able to get custom models working, but you'll have to follow the steps we took in the posts above. Let me know how I can best help!
Thomas
@thomas Also the model which @sansoy used was yolov5n, and I checked most of the code of inference helper it was related to yolov5, is the tflite server compatible to any yolo model?
Actually in a separate post I asked about object tracking with yolo models, from the article I read here and referring to the individual tracker documentations, somehow I need to have access to the tflite model to run these trackers, is there a way to integrate the trackers into the voxl-tflite-server, as the documentation says it provides hardware acceleration.
Implementing Object Tracking for Computer Vision (+ Code) | Datature Blog
The complete guide to compare the performance of multiple object tracking algorithms and build your object tracker project on Datature's platform.
(www.datature.io)
Understanding Multiple Object Tracking using DeepSORT
Get an overview of Tracking, the different types of object trackers, how tracking works, and what evaluation metrics are used to measure their performance.
LearnOpenCV – Learn OpenCV, PyTorch, Keras, Tensorflow with code, & tutorials (learnopencv.com)
-
@thomas Also the model which @sansoy used was yolov5n, and I checked most of the code of inference helper it was related to yolov5, is the tflite server compatible to any yolo model?
Actually in a separate post I asked about object tracking with yolo models, from the article I read here and referring to the individual tracker documentations, somehow I need to have access to the tflite model to run these trackers, is there a way to integrate the trackers into the voxl-tflite-server, as the documentation says it provides hardware acceleration.
Implementing Object Tracking for Computer Vision (+ Code) | Datature Blog
The complete guide to compare the performance of multiple object tracking algorithms and build your object tracker project on Datature's platform.
(www.datature.io)
Understanding Multiple Object Tracking using DeepSORT
Get an overview of Tracking, the different types of object trackers, how tracking works, and what evaluation metrics are used to measure their performance.
LearnOpenCV – Learn OpenCV, PyTorch, Keras, Tensorflow with code, & tutorials (learnopencv.com)
@thomas Edit about upgrading the sdk to nightly build: I tried it on my starling v2 drone, but somehow the nighly build is unable to write px4 parameters and the upgrade ends up failing (firmware issue link I reverted the sdk back to sdk 1.0 and the drone fortunately works again, but how would I avoid the number of classes bug which you highlighted above. I also have modified yolo to detect just one class.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login