Need help simulating .tflite yolo models on my linux machine.

[Ajay Bidyarthy]

@thomas
thanks for the above docs . So I have gone through the docs extensively but still dont find the instructions to deploy my custom tflite models on the VOXL2 mini chip. If possible can you please help me with the same. I would greatly appricate it

[A Former User]

@Ajay-Bidyarthy

Here's the docs section on custom models, give it a try and let me know how I can help.

https://docs.modalai.com/voxl-tflite-server/#custom-models

Thomas

[Ajay Bidyarthy]

@thomas
Thanks for this,
If you dont mind can we connect personally for few minutes? Also I wanted to ask that will we be able to simulate and benhcmark the yolo models without having camera sensor attached to the chip? If needed is there any way to test it using my macbook's camera?

Regards

[A Former User]

@Ajay-Bidyarthy

You actually may be able to benchmark voxl-tflite-server without an image sensor attached. The way we do this is with a tool called voxl-logger which lets us fully capture the inputs from the all the pipes over a given time period. If you had a log created by voxl-logger, you could then use voxl-replay to replay the log and then measure the performance of voxl-tflite-server over that interval. It would be much easier to simply use you VOXL with an image sensor though. If you want to go down the logger path, though, I can try to get a log together for you that you can replay. You definitely cannot test it with your Macbook camera.

Thanks,
Thomas

[Ajay Bidyarthy]

@thomas
Yes, thanks fro this,
I would appreicate if you can share the voxl-logger tool usage. Because right now when I just launch the tflite server by doing the configuration with my custom tflite model. it just says

=================================================================
skip_n_frames: 0

model: /usr/bin/dnn/yolov8n_model.tflite

input_pipe: /run/mpa/tracking/

delegate: gpu

allow_multiple: false

output_pipe_prefix: mobilenet

existing instance of voxl-tflite-server found, attempting to stop it
WARNING: Unknown model type provided! Defaulting post-process to object detection.
INFO: Created TensorFlow Lite delegate for GPU.
INFO: Initialized OpenCL-based API.
INFO: Created 1 GPU delegate kernels.

------VOXL TFLite Server------

Camera processing thread is now locked to the following cores: 4 5 6

And nothing afterwards. so i believe its running but I actually want to see the numbers. or the benchmark performances of the model. How do I do that.

Thanks a lot again,
Bhavya

[A Former User]

@Ajay-Bidyarthy

With all services on VOXL you should always run with the -h option if you're curious for more information on ways to use the service. In the case of voxl-tflite-server, we have both debug and timing options that can be specified to see printed outputs or timing benchmarks respectively.

Thomas

[Ajay Bidyarthy]

@thomas
Hi thomas,
Okay, so I actually did do those steps. And using that info I got to know that to benchmark the model I need to run

voxl-tflite-server

and then in another terminal I did:

voxl-logger -i imu1 -s 5000 --note "primary imu test"

but then i get this error:

[A Former User]

@Ajay-Bidyarthy

Really strange that you don't have the /data/modalai/ dir as that's where we store the sku.txt and other important info. Maybe try calibrating the IMU?

https://docs.modalai.com/calibrate-imu/

Thomas

[Ajay Bidyarthy]

@thomas
Yes, so we have the directory present, but it only has the sku.txt file there. I think its because of some .yml file being missing we get error. So what should we do to load that file successfully in the dir

[A Former User]

@Ajay-Bidyarthy

Try the IMU calibration steps I linked above.

Thomas

[Ajay Bidyarthy]

@thomas
Hi thomas,
So I did try calibrating the imu sensor using voxl-calibrate-imu. But I still dont see the .yml file in the data/modalai directory.
So I'll just share the brief overview on what steps I followed to get to the current point:

1. I connected VOXL (with only the default sensors, I did not attach any sensor seperately) to my computer.
2. I created a deb package to deploy to VOXL by loading my custom tflite files in the misc_files/usr/bin/dnn directory of the voxl-tflite-server with correspodning changs in the voxl-configure-tflite file present in the scripts/ qrb5165 folder. And then following the build and deploy commands to deploy the package to voxl
3. I then did adb shell to load into my VOXL and do see all the tflite files present there. I then ran voxl-configure-tflite command to load my custom tflite model.
4. Following that I simply executed voxl-tflite-server command
5. In another similar terminal I then tried benchmarking by doing voxl-logger command as shown and am now getting the error.
   file:///home/bhavya/Downloads/screen1.jpeg

file:///home/bhavya/Downloads/screen2.jpeg

Thanks

[A Former User]

@Ajay-Bidyarthy

Okay yeah I think I figured it out. Take a look at these lines from the voxl-logger code:

You can see that voxl-logger is just trying to copy over anything that matches *.yml . So while I work on a fix for this, could you just do inside your VOXL touch /data/modalai/something.yml. ? This should remove that error.

Keep me posted,
Thomas

[Ajay Bidyarthy]

@thomas
I attempted to use the voxl-logger by creating an empty something.yml file in the directory and executing the command. Despite encountering no errors during the execution, I did not receive any output beyond the line "connected to imu 0 sensor." My goal is to obtain benchmarking output similar to the tables shown in the ModalAI documentation: Benchmarks Documentation.

Could you kindly assist with this issue? I believe it would be more efficient if we could schedule a time for a discussion when you are available. This would allow us to address any questions or concerns promptly without the need for multiple back-and-forth messages.

Thank you for your time and assistance.

[Ajay Bidyarthy]

@thomas
Hi thomas,
Did you get a chance to check this issue? I am trying to benchmark the my models but I dont see any output or number I could use to summarize my benchmark or something.

[A Former User]

@Ajay-Bidyarthy

Yeah so when you run voxl-logger with -s 5000 you're running it for that many samples and it isn't going to have any printouts. I'm not sure what tables you're talking about with respect to the benchmarks documentation, maybe you could give me a link?

If you just want to collect a voxl-logger log, your commands above should work fine. You can see that it's creating logs in the /data/voxl-logger/ directory which you can replay with the voxl-logger command.

I help with the forum in addition to my work and so I don't have time for calls. If you're interested in setting up a bigger support effort, you can send an email to sales@modalai.com .

Hope this helps!

Thomas

[Ajay Bidyarthy]

@thomas
Hi thomas,
Yes, so the table I'm referring to is under the benchmark section here: https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks
So the thing is when I run the voxl-looger with -s 5000 I also expect the code to run for that many samples and output some log, but as you see from my screenshot, it just outputs ... ' samples: 0'

And no worries thomas, I do appreciate your time and help here

Regards

[A Former User]

@Ajay-Bidyarthy

Okay yeah let's back up here - why are we even trying to get a log with IMU samples in it? If you want to benchmark voxl-tflite-server, you don't need IMU samples to do so. What I had meant by all of this was: you don't have a camera attached to your VOXL if I remember correctly and so instead of attaching one, we could use a voxl-logger log which had camera data in it for replaying. This is because when you replay a voxl-logger log, it writes over all of the pipes. If we had a log which had camera data in it, you could replay it and your VOXL would "think" it had a camera attached and so then when you run voxl-tflite-server it could use the data coming from those pipes.

In this case, I probably need to provide you with a log that has camera data in it. No worries though, I have lots of logs and can upload one here. Do you have any preferences of what's in the log? I was just going to take my drone and wave it around for a few seconds but if you'd like something more we can do that. I'll also verify that this log can be used by tflite-server and what steps need to be taken.

If that seems reasonable let me know and I'll get started.

-Thomas

[Ajay Bidyarthy]

@thomas
Oh I understand now. So using your camera data voxl logs. We can simly use those logs(camera data) to benchmark tfliteserver and get the benchmark stats like shown here:
https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks

I would appreciate a lot if you could share them. Since it would be good to use those logs/same camera data to compare stats from diff custom tflite models

Thanks a lot

[A Former User]

@Ajay-Bidyarthy

Yep exactly! Let me get working on a log right now for you to use and then I'll upload it here and give you instructions on how to use it.

Thomas

[A Former User]

@Ajay-Bidyarthy

Here's a link to a .tar with a voxl-logger log that I recorded! https://drive.google.com/file/d/1c-snBLT6mmrwGpKE2WhVR-YQgumN0s6o/view?usp=sharing

You should use ADB to push this file onto VOXL and then use tar -xvf customlog.tar to unzip the file, it should unpack into /data/voxl-logger/log0009/. Then you should be able to voxl-replay -p /data/voxl-logger/log0009/. The only other important thing you need to do is edit voxl-tflite-server to predict from the tracking camera in /etc/modalai/voxl-tflite-server.conf.

Thomas