# Need help simulating .tflite yolo models on my linux machine.

Source: https://forum.modalai.com/topic/3103/need-help-simulating-tflite-yolo-models-on-my-linux-machine
Category: General Questions (https://forum.modalai.com/category/2/general-questions)
Posted: 2024-02-09 09:14:32 UTC by Ajay Bidyarthy
Replies: 56 · Views: 33248

## Ajay Bidyarthy · 2024-02-09 09:14:32 UTC

Hi, 
I'm new to VOXL. I have trained a yolo like model and have my yolo_custom.tflite model with me. How can I simulate it for a VOXL2mini environment. 
I tried following the steps there in https://docs.modalai.com/voxl-tflite-server/ But am lost at how to setup and configure and what all are the prerequisites.

## Reply by Guest · 2024-02-09 16:06:25 UTC

@Ajay-Bidyarthy 

Hey Ajay, happy to try and help! I wrote a custom section just on making and deploying custom models [here](https://docs.modalai.com/voxl-tflite-server/#custom-models), can I ask what parts specifically you're struggling with?

Thanks,
Thomas

## Reply by Ajay Bidyarthy · 2024-02-10 04:37:40 UTC (in reply to Guest)

@thomas 
Hey thomas, Thanks for the quick reply, 
So is it posible to simulate the tests in an environment without having the actual device with us? I'm asking this because you mentioned in some other thread that flashing the SDK on the device is necessary. I dont have the actual device with me right now and was hoping if I could somehow simulate the tflite models on my computer for testing purpose

## Reply by Guest · 2024-02-10 23:19:15 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

So it would certainly be best to have a device with you as we don't currently officially support the ability to run `voxl-tflite-server` locally or within a Docker container, for example. You could potentially find a way to make this work but it certainly isn't something that I've tested. 

Now you can still create .tflite models on your computer and you can still use them for inference locally, there are some guides on Google's TFLite documentation for how to do this. But inside of `voxl-tflite-server` we do some pre/post processing steps on the data which you would ultimately need to emulate to get a good idea of prediction metrics. This local inference, however, won't be useful for determining things like operational framerates for the model. For that, you'll definitely need a device to see how much the GPU/NPU can hardware accelerate the model you've created.

Hope this helps, happy to answer any more questions you may have!

Thomas Patton

## Reply by Ajay Bidyarthy · 2024-02-12 13:01:19 UTC (in reply to Guest)

@thomas 
Thank you for providing the necessary details. Once I acquire the device/chip for testing, I would appreciate it if you could also provide me with a comprehensive guide detailing the steps required to simulate and test TensorFlow Lite models in the VOXL environment. I have the tflite files ready with me. 
Thanks

## Reply by Guest · 2024-02-12 15:58:58 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

We have a documentation page on how to do that [here](https://docs.modalai.com/voxl-tflite-server/), give that a read and then we can talk about next steps!

Thomas Patton

## Reply by Ajay Bidyarthy · 2024-03-26 10:50:39 UTC (in reply to Guest)

@thomas 
Hi thomas,
Hope you're doing well,
So I have recieved my VOXL chip, and have followed the steps to setup, install and test my chip on my Macbook. I am able to launch voxl-docker image successfully as well using voxl-docker -i voxl-cross

But am completely lost as to how to launch and run my .tflite models from here onwards. Would appreciate if you can guide me through the same.

Thanks

## Reply by Guest · 2024-03-26 14:00:45 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Take a look at our [developer bootcamp](https://docs.modalai.com/voxl-developer-bootcamp/), you don't have to do the whole thing but it will get you used to interfacing with VOXL. Then when you feel ready check out the TFLite Server [docs](https://docs.modalai.com/voxl-tflite-server/), these detail how to load your custom .tflite models onto VOXL.

Thanks,

Thomas

## Reply by Ajay Bidyarthy · 2024-03-27 12:52:08 UTC

@thomas 
thanks for the above docs . So I have gone through the docs extensively but still dont find the instructions to deploy my custom tflite models on the VOXL2 mini chip. If possible can you please help me with the same. I would greatly appricate it

## Reply by Guest · 2024-03-27 16:32:11 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Here's the docs section on custom models, give it a try and let me know how I can help.

https://docs.modalai.com/voxl-tflite-server/#custom-models

Thomas

## Reply by Ajay Bidyarthy · 2024-03-28 05:18:39 UTC (in reply to Guest)

@thomas 
Thanks for this,
If you dont mind can we connect personally for few minutes? Also I wanted to ask that will we be able to simulate and benhcmark the yolo models without having camera sensor attached to the chip? If needed is there any way to test it using my macbook's camera?

Regards

## Reply by Guest · 2024-03-28 15:00:40 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

You actually may be able to benchmark voxl-tflite-server without an image sensor attached. The way we do this is with a tool called voxl-logger which lets us fully capture the inputs from the all the pipes over a given time period. If you had a log created by voxl-logger, you could then use voxl-replay to replay the log and then measure the performance of voxl-tflite-server over that interval. It would be much easier to simply use you VOXL with an image sensor though. If you want to go down the logger path, though, I can try to get a log together for you that you can replay. You definitely cannot test it with your Macbook camera.

Thanks,
Thomas

## Reply by Ajay Bidyarthy · 2024-03-29 06:34:32 UTC

@thomas 
Yes, thanks fro this,
I would appreicate if you can share the voxl-logger tool usage. Because right now when I just launch the tflite server by doing the configuration with my custom tflite model. it just says

=================================================================
skip_n_frames:                    0
=================================================================
model:                            /usr/bin/dnn/yolov8n_model.tflite
=================================================================
input_pipe:                       /run/mpa/tracking/
=================================================================
delegate:                         gpu
=================================================================
allow_multiple:                   false
=================================================================
output_pipe_prefix:               mobilenet
=================================================================
existing instance of voxl-tflite-server found, attempting to stop it
WARNING: Unknown model type provided! Defaulting post-process to object detection.
INFO: Created TensorFlow Lite delegate for GPU.
INFO: Initialized OpenCL-based API.
INFO: Created 1 GPU delegate kernels.

------VOXL TFLite Server------

Camera processing thread is now locked to the following cores: 4 5 6

And nothing afterwards. so i believe its running but I actually want to see the numbers. or the benchmark performances of the model. How do I do that.

Thanks a lot again,
Bhavya

## Reply by Guest · 2024-03-29 14:53:00 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

With all services on VOXL you should always run with the `-h` option if you're curious for more information on ways to use the service. In the case of `voxl-tflite-server`, we have both debug and timing options that can be specified to see printed outputs or timing benchmarks respectively.

Thomas

## Reply by Ajay Bidyarthy · 2024-03-29 17:02:30 UTC (in reply to Guest)

@thomas 
Hi thomas, 
Okay, so I actually did do those steps. And using that info I got to know that to benchmark the model I need to run 
>> voxl-tflite-server

and then in another terminal I did:
>> voxl-logger -i imu1 -s 5000 --note "primary imu test"

but then i get this error:
![d37108ef-5790-4783-bdab-339cbfc23b77-image.png](https://forum.modalai.com/assets/uploads/files/1711731736774-d37108ef-5790-4783-bdab-339cbfc23b77-image.png)

## Reply by Guest · 2024-03-29 18:21:30 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Really strange that you don't have the `/data/modalai/` dir as that's where we store the `sku.txt` and other important info. Maybe try calibrating the IMU?

https://docs.modalai.com/calibrate-imu/

Thomas

## Reply by Ajay Bidyarthy · 2024-03-30 05:32:00 UTC (in reply to Guest)

@thomas 
Yes, so we have the directory present, but it only has the sku.txt file there. I think its because of some .yml file being missing we get error. So what should we do to load that file successfully in the dir

## Reply by Guest · 2024-03-31 18:01:33 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Try the IMU calibration steps I linked above.

Thomas

## Reply by Ajay Bidyarthy · 2024-04-01 06:09:20 UTC (in reply to Guest)

@thomas 
Hi thomas,
So I did try calibrating the imu sensor using voxl-calibrate-imu. But I still dont see the .yml file in the data/modalai directory. 
So I'll just share the brief overview on what steps I followed to get to the current point:
1. I connected VOXL (with only the default sensors, I did not attach any sensor seperately) to my computer.
2. I created a deb package to deploy to VOXL by loading my custom tflite files in the misc_files/usr/bin/dnn directory of the voxl-tflite-server with correspodning changs in the voxl-configure-tflite file present in the scripts/ qrb5165 folder. And then following the build and deploy commands to deploy the package to voxl
3. I then did adb shell to load into my VOXL and do see all the tflite files present there. I then ran voxl-configure-tflite command to load my custom tflite model.
4. Following that I simply executed voxl-tflite-server command
5. In another similar terminal I then tried benchmarking by doing voxl-logger command as shown and am now getting the error.
![876ad166-0ed6-40d5-8d86-6f61b06e991e-screen1.jpeg](https://forum.modalai.com/assets/uploads/files/1711951719556-876ad166-0ed6-40d5-8d86-6f61b06e991e-screen1.jpeg) file:///home/bhavya/Downloads/screen1.jpeg

![0060900e-6d63-41bf-8c56-85a0d5f00bb2-screen2.jpeg](https://forum.modalai.com/assets/uploads/files/1711951731644-0060900e-6d63-41bf-8c56-85a0d5f00bb2-screen2.jpeg) file:///home/bhavya/Downloads/screen2.jpeg

Thanks

## Reply by Guest · 2024-04-01 20:09:26 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Okay yeah I think I figured it out. Take a look at these lines from the `voxl-logger` code: 

![17498768-d79e-438f-94f5-3424651307ba-image.png](https://forum.modalai.com/assets/uploads/files/1712001843596-17498768-d79e-438f-94f5-3424651307ba-image.png) 

You can see that `voxl-logger` is just trying to copy over **anything** that matches *.yml . So while I work on a fix for this, could you just do inside your VOXL `touch /data/modalai/something.yml`. ? This should remove that error.

Keep me posted,
Thomas

## Reply by Ajay Bidyarthy · 2024-04-02 05:40:56 UTC (in reply to Guest)

@thomas
I attempted to use the voxl-logger by creating an empty something.yml file in the directory and executing the command. Despite encountering no errors during the execution, I did not receive any output beyond the line "connected to imu 0 sensor." My goal is to obtain benchmarking output similar to the tables shown in the ModalAI documentation: Benchmarks Documentation.

Could you kindly assist with this issue? I believe it would be more efficient if we could schedule a time for a discussion when you are available. This would allow us to address any questions or concerns promptly without the need for multiple back-and-forth messages.

Thank you for your time and assistance.

 ![b15a6593-d906-4dfb-b72f-2cbe02ea7f0b-image.png](https://forum.modalai.com/assets/uploads/files/1712036403194-b15a6593-d906-4dfb-b72f-2cbe02ea7f0b-image.png)

## Reply by Ajay Bidyarthy · 2024-04-02 17:22:40 UTC (in reply to Guest)

@thomas
Hi thomas,
Did you get a chance to check this issue? I am trying to benchmark the my models but I dont see any output or number I could use to summarize my benchmark or something.

## Reply by Guest · 2024-04-02 17:33:49 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Yeah so when you run voxl-logger with -s 5000 you're running it for that many samples and it isn't going to have any printouts. I'm not sure what tables you're talking about with respect to the benchmarks documentation, maybe you could give me a link?

If you just want to collect a voxl-logger log, your commands above should work fine. You can see that it's creating logs in the /data/voxl-logger/ directory which you can replay with the voxl-logger command.

I help with the forum in addition to my work and so I don't have time for calls. If you're interested in setting up a bigger support effort, you can send an email to sales@modalai.com .

Hope this helps!

Thomas

## Reply by Ajay Bidyarthy · 2024-04-02 18:10:53 UTC (in reply to Guest)

@thomas 
Hi thomas, 
Yes, so the table I'm referring to is under the benchmark section here: https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks 
So the thing is when I run the voxl-looger with -s 5000 I also expect the code to run for that many samples and output some log, but as you see from my screenshot, it just outputs ... ' samples:    0'

And no worries thomas, I do appreciate your time and help here :)

Regards

## Reply by Guest · 2024-04-02 18:21:41 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Okay yeah let's back up here - why are we even trying to get a log with IMU samples in it? If you want to benchmark `voxl-tflite-server`, you don't need IMU samples to do so. What I had meant by all of this was: you don't have a camera attached to your VOXL if I remember correctly and so instead of attaching one, we could use a `voxl-logger` log which had camera data in it for replaying. This is because when you replay a `voxl-logger` log, it writes over all of the pipes. If we had a log which had camera data in it, you could replay it and your VOXL would "think" it had a camera attached and so then when you run `voxl-tflite-server` it could use the data coming from those pipes.

In this case, I probably need to provide you with a log that has camera data in it. No worries though, I have lots of logs and can upload one here. Do you have any preferences of what's in the log? I was just going to take my drone and wave it around for a few seconds but if you'd like something more we can do that. I'll also verify that this log can be used by tflite-server and what steps need to be taken.

If that seems reasonable let me know and I'll get started.

-Thomas

## Reply by Ajay Bidyarthy · 2024-04-02 18:27:29 UTC (in reply to Guest)

@thomas 
Oh I understand now. So using your camera data voxl logs. We can simly use those logs(camera data) to benchmark tfliteserver and get the benchmark stats like shown here:
https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks

I would appreciate a lot if you could share them. Since it would be good to use those logs/same camera data to compare stats from diff custom tflite models 

Thanks a lot

## Reply by Guest · 2024-04-02 18:32:47 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Yep exactly! Let me get working on a log right now for you to use and then I'll upload it here and give you instructions on how to use it.

Thomas

## Reply by Guest · 2024-04-02 18:48:11 UTC (in reply to Guest)

@Ajay-Bidyarthy 

Here's a link to a .tar with a voxl-logger log that I recorded! https://drive.google.com/file/d/1c-snBLT6mmrwGpKE2WhVR-YQgumN0s6o/view?usp=sharing

You should use ADB to push this file onto VOXL and then use `tar -xvf customlog.tar` to unzip the file, it should unpack into `/data/voxl-logger/log0009/`. Then you should be able to `voxl-replay -p /data/voxl-logger/log0009/`. The only other important thing you need to do is edit `voxl-tflite-server` to predict from the `tracking` camera in `/etc/modalai/voxl-tflite-server.conf`.

Thomas

## Reply by Ajay Bidyarthy · 2024-04-03 05:25:12 UTC (in reply to Guest)

@thomas 
Hey thomas,
So I have followed your steps and loaded the log0009 data and then ran voxl-replay command while running voxl-tflite-server correctly configured with tracking option. But nothing happens after that it seems. 
![9a3d0154-77d3-41bb-89ae-6401f5a63446-image.png](https://forum.modalai.com/assets/uploads/files/1712121748470-9a3d0154-77d3-41bb-89ae-6401f5a63446-image.png) 

So the output I'm expecting at the end is to be able to have something like
Model	Task	Avg Cpu Inference(ms)	Avg Gpu Inference(ms)	Avg NNAPI Inference(ms)	Max Frames Per Second(fps)	Input Dimensions	Source
custom-tflite-model	Object Detection	33.89ms	24.68ms	34.42ms	34.86750349	[1,300,300,3]

Where can I get such output after running the voxl-replay command. I think I'm missing some steps after that.

Please let me know what I'm missing here.
Thanks

## Reply by Guest · 2024-04-03 15:20:07 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Yeah, so my default all of our services don't print anything out when run. Do `voxl-tflite-server -t` for timing printouts.

This won't give you a super organized printout like what you are writing. But it will give you out data on the per-iteration callback times and the breakdown per function. You'll then need to modify `/etc/modalai/voxl-tflite-server.conf` in different ways to get different timing printouts in order to build up a table of benchmarks.

Thomas

## Reply by Ajay Bidyarthy · 2024-04-04 08:12:34 UTC (in reply to Guest)

@thomas 
Okay, So i did run voxl-tflite-server -t alongside voxl-replay command but am still not getting anything. 
For yolov5 I get no output after running voxl replay. it just outputs this:
![466ee992-244f-43e2-9ea8-1a945a8028c7-image.png](https://forum.modalai.com/assets/uploads/files/1712217931895-466ee992-244f-43e2-9ea8-1a945a8028c7-image.png)  

For my custom yolov7 nd yolov8 I get these errors:
![a1893cc6-6546-481b-8bda-58c64dbb32cc-image.png](https://forum.modalai.com/assets/uploads/files/1712217997377-a1893cc6-6546-481b-8bda-58c64dbb32cc-image.png) 
![ba5b55d7-22dd-4b75-87dc-d03c9b6c80d6-image.png](https://forum.modalai.com/assets/uploads/files/1712218005812-ba5b55d7-22dd-4b75-87dc-d03c9b6c80d6-image.png) 

I even referred to this thread https://forum.modalai.com/topic/2997/can-anyone-recommend-a-tflite-colab-notebook-for-voxl2-training/3

and configured the main.cpp file by adding if else statement for custom model to use YOLO model instead of OBJECT_DETECTION as the default one before even running the above commands.

I'm not sure why am I still not getting any output.
Thanks

## Reply by Guest · 2024-04-04 14:41:46 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy I'll look into it today.

Thomas

## Reply by Guest · 2024-04-04 21:42:12 UTC (in reply to Guest)

@thomas @Ajay-Bidyarthy 

Sadly I won't have time to look at this today and am out of office tomorrow so I can't fully debug this for you. Let's focus just on YOLOv5 right now since that's the one we have onboard by default and it isn't segfaulting. Make sure you're launching tflite-server in a terminal and then in a separate terminal running the voxl-replay command. 

I apologize for this, sorry I didn't have time for it today. 

Thomas

## Reply by Ajay Bidyarthy · 2024-04-05 09:48:17 UTC (in reply to Guest)

@thomas 
No worries. Ok, but even for yolov5 I've been running both voxl replay and tflite server commands in separate windows/terminals but still facing the issue.

Let me know why is it so and how can we resolve this.
Thanks

## Reply by Guest · 2024-04-08 15:48:35 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Hey Ajay, working on it right now. Will let you know when I have an update.

Thomas

## Reply by Guest · 2024-04-08 16:42:26 UTC (in reply to Guest)

@thomas @Ajay-Bidyarthy 

Okay so I've identified a problem in our timing code and am working on a fix. Here's a short-term solution for you:

1. Make sure `/etc/modalai/voxl-tflite-server.conf` has `"input_pipe"` set to `"/run/mpa/tracking"`
2. Get your VOXL's IP address with `voxl-my-ip` and enter this IP into your web browser to pull up voxl-portal in your browser.
3. Run `voxl-tflite-server` on VOXL
4. Run your voxl-logger log as `voxl-logger -p /data/voxl-logger/log0009/` or whatever the path is saved as

Now in your web browser, from the "Cameras" dropdown click tflite. You may need to refresh the page to get this to pop up. The permalink to this page is `<YOUR IP>/video.html?cam=tflite` (e.g. `http://192.168.0.198/video.html?cam=tflite`) so you can optionally just go there. This overlay shows the ML model running and in the upper left of the frame shows a framerate. You can use this to estimate for the time being.

Am working on the fix now, will let you know when it's released.

Thomas

## Reply by Guest · 2024-04-08 17:51:25 UTC (in reply to Guest)

@thomas @Ajay-Bidyarthy 

Okay, I've pushed up a change for our nightly builds to fix the timing printouts with the `-t` option. So starting **tomorrow** you can flash the latest nightly SDK build and have this fix. If you'd like it today let me know and I can push the .deb up to Google Drive so you can flash it directly. [Here](https://console.cloud.google.com/storage/browser/platform-nightlies/voxl2;tab=objects?organizationId=517175400245&project=modalai-core-services&prefix=&forceOnObjectsSortingFiltering=true&pageState=(%22StorageObjectListTable%22:(%22s%22:%5B(%22i%22:%22objectListDisplayFields%2FtimeCreated%22,%22s%22:%221%22),(%22i%22:%22displayName%22,%22s%22:%220%22)%5D))) is the link for the nightly SDK builds.

Thomas

## Reply by Ajay Bidyarthy · 2024-04-09 05:16:56 UTC (in reply to Guest)

@thomas 
Hi,
Thanks for this info. So I have tried accessing the portal by running the voxl-my-ip command but found nothing getting printed out when I run that command. I inspected to make sure my voxl-portal service is running perfectly but still cant seem to get the ip for portal.
Also, can you push the latest SDK as well or share the deb with me.
![99a97d0c-8d77-41b3-9de3-e95e59c78cf6-WhatsApp Image 2024-04-09 at 10.46.27 AM.jpeg](https://forum.modalai.com/assets/uploads/files/1712639807096-99a97d0c-8d77-41b3-9de3-e95e59c78cf6-whatsapp-image-2024-04-09-at-10.46.27-am.jpeg) file:///home/bhavya/Downloads/WhatsApp%20Image%202024-04-09%20at%2010.46.27%20AM.jpeg

Thanks

## Reply by Guest · 2024-04-09 14:35:24 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Make sure you set up WiFi on VOXL! Here's the guide: https://docs.modalai.com/voxl-2-wifi-setup/

Thomas

## Reply by Ajay Bidyarthy · 2024-04-12 06:39:37 UTC (in reply to Guest)

@thomas 
Hi thomas,
Could you please provide us the latest nightly sdk build for VOXL2 mini if you think we can work with that for now?

Regards,
OKSI Team

## Reply by Guest · 2024-04-12 14:53:54 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

https://console.cloud.google.com/storage/browser/_details/platform-nightlies/voxl2/voxl2_SDK_nightly_20240411.tar.gz;tab=live_object?organizationId=517175400245&project=modalai-core-services

## Reply by Ajay Bidyarthy · 2024-04-15 08:01:22 UTC (in reply to Guest)

@thomas 
Hi thomas,
Is this package for voxl2min or voxl2?
Regards

## Reply by Ajay Bidyarthy · 2024-04-15 08:32:03 UTC (in reply to Ajay Bidyarthy)

@thomas 
Because I think this SDK is meant for M-0054 while our version is M0104. Let me know if we can go ahead and flash this SDK or not?
Thanks

## Reply by Ajay Bidyarthy · 2024-04-15 10:53:37 UTC (in reply to Guest)

@thomas 
Also, Now that I have flashed the SDK onto my chip its showing voxl2 in the shell screen instead of voxl2-mini. And even if I try and go ahead and do the following steps. I still cant get the desired output.
![2ab88265-965c-4de3-a187-2ecc2d5f3cce-Screenshot from 2024-04-15 14-29-02.png](https://forum.modalai.com/assets/uploads/files/1713178411751-2ab88265-965c-4de3-a187-2ecc2d5f3cce-screenshot-from-2024-04-15-14-29-02.png) 
![135af24f-36e9-44ce-b42e-5a6ac8041a2d-image.png](https://forum.modalai.com/assets/uploads/files/1713178382710-135af24f-36e9-44ce-b42e-5a6ac8041a2d-image.png) file:///home/bhavya/Pictures/Screenshots/Screenshot%20from%202024-04-15%2014-29-02.png

## Reply by Guest · 2024-04-15 15:34:23 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Oh my apologies, here's the link for VOXL2 Mini nightly SDK. I almost never work with these so apologies for the confusion

https://console.cloud.google.com/storage/browser/_details/platform-nightlies/voxl2-mini/voxl2-mini_SDK_nightly_20240412.tar.gz;tab=live_object?organizationId=517175400245&project=modalai-core-services

Thomas

## Reply by Ajay Bidyarthy · 2024-04-16 10:07:58 UTC (in reply to Guest)

@thomas 
Hey thomas, 
So I tried with this new sdk. Unfortunately, I still dont see any output here as well
![546cef03-4162-442d-93fc-ce6600d564d1-image.png](https://forum.modalai.com/assets/uploads/files/1713262068132-546cef03-4162-442d-93fc-ce6600d564d1-image.png)

## Reply by Guest · 2024-04-16 15:52:00 UTC (in reply to Guest)

Have you also tried using this method? Or were there issues getting WiFi setup?

@thomas said in [Need help simulating \.tflite yolo models on my linux machine\.](/post/16045):
> Okay so I've identified a problem in our timing code and am working on a fix. Here's a short-term solution for you:
> 
>     Make sure /etc/modalai/voxl-tflite-server.conf has "input_pipe" set to "/run/mpa/tracking"
>     Get your VOXL's IP address with voxl-my-ip and enter this IP into your web browser to pull up voxl-portal in your browser.
>     Run voxl-tflite-server on VOXL
>     Run your voxl-logger log as voxl-logger -p /data/voxl-logger/log0009/ or whatever the path is saved as
> 
> Now in your web browser, from the "Cameras" dropdown click tflite. You may need to refresh the page to get this to pop up. The permalink to this page is <YOUR IP>/video.html?cam=tflite (e.g. http://192.168.0.198/video.html?cam=tflite) so you can optionally just go there. This overlay shows the ML model running and in the upper left of the frame shows a framerate. You can use this to estimate for the time being.
> 
> Am working on the fix now, will let you know when it's released.
> 
> Thomas

Yeah I'm really not sure what's going on here, on that nightly SDK `voxl-tflite-server -t` is printing out timing information. I don't have a VOXL2 Mini to test on so if it's a specific issue there I'm really not sure. Let me try some stuff today and I'll get back to you.

Thomas

## Reply by Ajay Bidyarthy · 2024-04-16 19:14:06 UTC (in reply to Guest)

@thomas 
Hi thomas, 
Yes so right now we tried figuring out if maybe the issue is with the model not being specific to voxl2mini or maybe the logs are generated based on voxl2. But I dont think thats the case. 
The thing which I'm concerned about is:
1. How do we identify or pinpoint the problem ...like are we supposed to see the output after the 'Disconnected from camera server' line or before? And if we are supposed to see it and are not able to... then where exactly is the problem happening in the src/main.cpp file of voxl-tflite-server. Can we debug that somehow?
2. Also did you check running the voxl-replay logger command directly without having any camera setup or camera-server configured.... similar to the way we are currently running on our end.

## Reply by Guest · 2024-04-16 23:17:56 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

If `voxl-tflite-server` was running into an actual error you'd see some sort of an error message printout. We typically debug issues through `voxl-portal` but if you aren't able to get WiFi set up you won't be able to do so. I again reflashed a nightly SDK to confirm things are working on my end and they are. To reconfirm:
- Flash the latest SDK nightly from the link shared above
- Modify `/etc/modalai/voxl-tflite-server.conf` to have `/run/mpa/tracking` as its input. This file gets overwritten on flash so make sure you do this.
- `voxl-tflite-server -t` in one terminal
- `voxl-replay -p /data/voxl-logger/log0009/` in another

I can tell you that replay doesn't care about camera server being configured, it should work regardless.

-Thomas

## Reply by Ajay Bidyarthy · 2024-04-17 08:23:22 UTC (in reply to Guest)

@thomas 
Hi thomas,
So I followed just the 4 steps you shared and its working perfectly fine with the default models. 
![9cde7d30-94c5-4b41-9fff-179560d17050-Screenshot from 2024-04-17 12-28-53.png](https://forum.modalai.com/assets/uploads/files/1713342574047-9cde7d30-94c5-4b41-9fff-179560d17050-screenshot-from-2024-04-17-12-28-53.png) 
But when i deploy my custom voxl-tflite-server deb package onto the chip it says warning downgrading from voxl-tflite-server 0.3.3 to 0.3.2. So maybe I feel the issue is with the deb package of voxl-tflite-server I'm building with my custom tflite models and building the package out of it and deploying. 
Maybe you can update the gitlab repo so that I can build my custom voxl-tflite-server deb package and deploy it. 
Let me know if you can do this.

Thanks
file:///home/bhavya/Pictures/Screenshots/Screenshot%20from%202024-04-17%2012-28-53.png

## Reply by Guest · 2024-04-17 19:21:34 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Hey Ajay,

The Gitlab repo for tflite server actually is updated, just only for the `dev` branch. We're not quite ready to merge this to `master` yet. So you can just switch to that branch and branch your changes off of that and you should be fine.

Thanks,
Thomas

## Reply by Ajay Bidyarthy · 2024-04-18 12:50:04 UTC (in reply to Guest)

@thomas 
Hi thomas,
So we were able to run our custom tflite models using the updated tflite-server 0.3.3 . But now we are facing some new problem which is related to our custom tflite files. We are getting Segmentation fault errors as shown.
![f1398dbc-5b50-4e4f-bf7b-61d5b9d1c615-image.png](https://forum.modalai.com/assets/uploads/files/1713444225787-f1398dbc-5b50-4e4f-bf7b-61d5b9d1c615-image.png) 
![d5d6a388-25e2-4199-b528-103daa493311-image.png](https://forum.modalai.com/assets/uploads/files/1713444235793-d5d6a388-25e2-4199-b528-103daa493311-image.png) 
 Could you help us regarding what exactly could we be missing here. Since I followed the steps mentioned in the documentation to create the compatible tflite models (using fp16)
https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks

Is there something we can test externally to check if our tflite model is compatible or not? Like some code or tool.

Regards,

## Reply by Guest · 2024-04-18 15:33:09 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

The first question I usually ask people about custom models is the labels. Are you using the same classes as the original YOLO or different classes? If you have a different labels file, that needs to be reflected in `/usr/bin/dinn` in the corresponding .txt file. 

Also is this YOLOv8? 

Thanks,
Thomas

## Reply by Ajay Bidyarthy · 2024-04-18 18:40:39 UTC (in reply to Guest)

@thomas 
Hi thomas,
So I was assuming the labels which are used for yolov5 which is present in the repo by default would work fine with custom models(yolov7 and yolov8) as well.

And yes the last screenshot is for yolov8.

## Reply by Guest · 2024-04-18 19:53:46 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

It's dependent on what labels/classes you trained your YOLOv7/8 networks on. If it's on the same labels as whats in `/usr/bin/dnn/yolov5_labels.txt` then you should be fine. If not you could definitely segfault.

If that's all fine then I'm not immediately aware what could be causing an issue. As you likely know we don't officially support YOLOv7/8 and so you'll need to debug it yourself to find the issue. If you do find an issue and have a corresponding fix, you're more than welcome to open up a Merge Request in the repo and I can take a look at it. We've wanted to add support for YOLOv7/8 for some time but it just isn't a high enough priority at the moment.

Hope this helps,

Thomas

## Reply by Ajay Bidyarthy · 2024-04-22 20:18:40 UTC (in reply to Guest)

@thomas 
Hey thomas,
Hope you're doing well,
So I have a question - The three lines of results which we get after running the models :
1. Preprocessing time , 2. Inference Time, 3. Post processing Time
Are they specific to CPU only. or are they according to if the task is delgated to GPU or something.

If its the later case, then can we manually run the tflite-server by manually delegating the model to the GPU or DPU, instead of it automatically detecting and processing accordingly.
Regards,
Bhavya

## Reply by Guest · 2024-04-24 15:40:02 UTC (in reply to Ajay Bidyarthy)

@Ajay-Bidyarthy 

Only the inference time should be affected by your choice of hardware. The other two should be the same regardless of your choice.

Thomas