Need help simulating .tflite yolo models on my linux machine.

So it would certainly be best to have a device with you as we don't currently officially support the ability to run voxl-tflite-server locally or within a Docker container, for example. You could potentially find a way to make this work but it certainly isn't something that I've tested.

Now you can still create .tflite models on your computer and you can still use them for inference locally, there are some guides on Google's TFLite documentation for how to do this. But inside of voxl-tflite-server we do some pre/post processing steps on the data which you would ultimately need to emulate to get a good idea of prediction metrics. This local inference, however, won't be useful for determining things like operational framerates for the model. For that, you'll definitely need a device to see how much the GPU/NPU can hardware accelerate the model you've created.

Hope this helps, happy to answer any more questions you may have!

Thomas Patton

Ajay Bidyarthy

@thomas
Thank you for providing the necessary details. Once I acquire the device/chip for testing, I would appreciate it if you could also provide me with a comprehensive guide detailing the steps required to simulate and test TensorFlow Lite models in the VOXL environment. I have the tflite files ready with me.
Thanks

@Ajay-Bidyarthy

We have a documentation page on how to do that here, give that a read and then we can talk about next steps!

Thomas Patton

Ajay Bidyarthy

@thomas
Hi thomas,
Hope you're doing well,
So I have recieved my VOXL chip, and have followed the steps to setup, install and test my chip on my Macbook. I am able to launch voxl-docker image successfully as well using voxl-docker -i voxl-cross

But am completely lost as to how to launch and run my .tflite models from here onwards. Would appreciate if you can guide me through the same.

Thanks

@Ajay-Bidyarthy

Take a look at our developer bootcamp, you don't have to do the whole thing but it will get you used to interfacing with VOXL. Then when you feel ready check out the TFLite Server docs, these detail how to load your custom .tflite models onto VOXL.

Thanks,

Thomas

Ajay Bidyarthy

@thomas
thanks for the above docs . So I have gone through the docs extensively but still dont find the instructions to deploy my custom tflite models on the VOXL2 mini chip. If possible can you please help me with the same. I would greatly appricate it

@Ajay-Bidyarthy

Here's the docs section on custom models, give it a try and let me know how I can help.

https://docs.modalai.com/voxl-tflite-server/#custom-models

Thomas

Ajay Bidyarthy

@thomas
Thanks for this,
If you dont mind can we connect personally for few minutes? Also I wanted to ask that will we be able to simulate and benhcmark the yolo models without having camera sensor attached to the chip? If needed is there any way to test it using my macbook's camera?

Regards

@Ajay-Bidyarthy

You actually may be able to benchmark voxl-tflite-server without an image sensor attached. The way we do this is with a tool called voxl-logger which lets us fully capture the inputs from the all the pipes over a given time period. If you had a log created by voxl-logger, you could then use voxl-replay to replay the log and then measure the performance of voxl-tflite-server over that interval. It would be much easier to simply use you VOXL with an image sensor though. If you want to go down the logger path, though, I can try to get a log together for you that you can replay. You definitely cannot test it with your Macbook camera.

Thanks,
Thomas

Ajay Bidyarthy

@thomas
Yes, thanks fro this,
I would appreicate if you can share the voxl-logger tool usage. Because right now when I just launch the tflite server by doing the configuration with my custom tflite model. it just says

=================================================================
skip_n_frames: 0

model: /usr/bin/dnn/yolov8n_model.tflite

input_pipe: /run/mpa/tracking/

delegate: gpu

allow_multiple: false

output_pipe_prefix: mobilenet

existing instance of voxl-tflite-server found, attempting to stop it
WARNING: Unknown model type provided! Defaulting post-process to object detection.
INFO: Created TensorFlow Lite delegate for GPU.
INFO: Initialized OpenCL-based API.
INFO: Created 1 GPU delegate kernels.

------VOXL TFLite Server------

Camera processing thread is now locked to the following cores: 4 5 6

And nothing afterwards. so i believe its running but I actually want to see the numbers. or the benchmark performances of the model. How do I do that.

Thanks a lot again,
Bhavya

@Ajay-Bidyarthy

With all services on VOXL you should always run with the -h option if you're curious for more information on ways to use the service. In the case of voxl-tflite-server, we have both debug and timing options that can be specified to see printed outputs or timing benchmarks respectively.

Thomas

Ajay Bidyarthy

@thomas
Hi thomas,
Okay, so I actually did do those steps. And using that info I got to know that to benchmark the model I need to run

voxl-tflite-server

and then in another terminal I did:

voxl-logger -i imu1 -s 5000 --note "primary imu test"

but then i get this error:

@Ajay-Bidyarthy

Really strange that you don't have the /data/modalai/ dir as that's where we store the sku.txt and other important info. Maybe try calibrating the IMU?

https://docs.modalai.com/calibrate-imu/

Thomas

Ajay Bidyarthy

@thomas
Yes, so we have the directory present, but it only has the sku.txt file there. I think its because of some .yml file being missing we get error. So what should we do to load that file successfully in the dir

@Ajay-Bidyarthy

Try the IMU calibration steps I linked above.

Thomas

Ajay Bidyarthy

@thomas
Hi thomas,
So I did try calibrating the imu sensor using voxl-calibrate-imu. But I still dont see the .yml file in the data/modalai directory.
So I'll just share the brief overview on what steps I followed to get to the current point:

I connected VOXL (with only the default sensors, I did not attach any sensor seperately) to my computer.
I created a deb package to deploy to VOXL by loading my custom tflite files in the misc_files/usr/bin/dnn directory of the voxl-tflite-server with correspodning changs in the voxl-configure-tflite file present in the scripts/ qrb5165 folder. And then following the build and deploy commands to deploy the package to voxl
I then did adb shell to load into my VOXL and do see all the tflite files present there. I then ran voxl-configure-tflite command to load my custom tflite model.
Following that I simply executed voxl-tflite-server command
In another similar terminal I then tried benchmarking by doing voxl-logger command as shown and am now getting the error.
file:///home/bhavya/Downloads/screen1.jpeg

file:///home/bhavya/Downloads/screen2.jpeg

Thanks

@Ajay-Bidyarthy

Okay yeah I think I figured it out. Take a look at these lines from the voxl-logger code:

You can see that voxl-logger is just trying to copy over anything that matches *.yml . So while I work on a fix for this, could you just do inside your VOXL touch /data/modalai/something.yml. ? This should remove that error.

Keep me posted,
Thomas

Ajay Bidyarthy

@thomas
I attempted to use the voxl-logger by creating an empty something.yml file in the directory and executing the command. Despite encountering no errors during the execution, I did not receive any output beyond the line "connected to imu 0 sensor." My goal is to obtain benchmarking output similar to the tables shown in the ModalAI documentation: Benchmarks Documentation.

Could you kindly assist with this issue? I believe it would be more efficient if we could schedule a time for a discussion when you are available. This would allow us to address any questions or concerns promptly without the need for multiple back-and-forth messages.

Thank you for your time and assistance.

Ajay Bidyarthy

@thomas
Hi thomas,
Did you get a chance to check this issue? I am trying to benchmark the my models but I dont see any output or number I could use to summarize my benchmark or something.

@Ajay-Bidyarthy

Yeah so when you run voxl-logger with -s 5000 you're running it for that many samples and it isn't going to have any printouts. I'm not sure what tables you're talking about with respect to the benchmarks documentation, maybe you could give me a link?

If you just want to collect a voxl-logger log, your commands above should work fine. You can see that it's creating logs in the /data/voxl-logger/ directory which you can replay with the voxl-logger command.

I help with the forum in addition to my work and so I don't have time for calls. If you're interested in setting up a bigger support effort, you can send an email to sales@modalai.com .

Hope this helps!

Thomas

Need help simulating .tflite yolo models on my linux machine.

================================================================= skip_n_frames: 0

model: /usr/bin/dnn/yolov8n_model.tflite

input_pipe: /run/mpa/tracking/

delegate: gpu

allow_multiple: false

output_pipe_prefix: mobilenet

=================================================================
skip_n_frames: 0