Need help simulating .tflite yolo models on my linux machine.

With all services on VOXL you should always run with the -h option if you're curious for more information on ways to use the service. In the case of voxl-tflite-server, we have both debug and timing options that can be specified to see printed outputs or timing benchmarks respectively.

Thomas

Ajay Bidyarthy

@thomas
Hi thomas,
Okay, so I actually did do those steps. And using that info I got to know that to benchmark the model I need to run

voxl-tflite-server

and then in another terminal I did:

voxl-logger -i imu1 -s 5000 --note "primary imu test"

but then i get this error:

@Ajay-Bidyarthy

Really strange that you don't have the /data/modalai/ dir as that's where we store the sku.txt and other important info. Maybe try calibrating the IMU?

https://docs.modalai.com/calibrate-imu/

Thomas

Ajay Bidyarthy

@thomas
Yes, so we have the directory present, but it only has the sku.txt file there. I think its because of some .yml file being missing we get error. So what should we do to load that file successfully in the dir

@Ajay-Bidyarthy

Try the IMU calibration steps I linked above.

Thomas

Ajay Bidyarthy

@thomas
Hi thomas,
So I did try calibrating the imu sensor using voxl-calibrate-imu. But I still dont see the .yml file in the data/modalai directory.
So I'll just share the brief overview on what steps I followed to get to the current point:

I connected VOXL (with only the default sensors, I did not attach any sensor seperately) to my computer.
I created a deb package to deploy to VOXL by loading my custom tflite files in the misc_files/usr/bin/dnn directory of the voxl-tflite-server with correspodning changs in the voxl-configure-tflite file present in the scripts/ qrb5165 folder. And then following the build and deploy commands to deploy the package to voxl
I then did adb shell to load into my VOXL and do see all the tflite files present there. I then ran voxl-configure-tflite command to load my custom tflite model.
Following that I simply executed voxl-tflite-server command
In another similar terminal I then tried benchmarking by doing voxl-logger command as shown and am now getting the error.
file:///home/bhavya/Downloads/screen1.jpeg

file:///home/bhavya/Downloads/screen2.jpeg

Thanks

@Ajay-Bidyarthy

Okay yeah I think I figured it out. Take a look at these lines from the voxl-logger code:

You can see that voxl-logger is just trying to copy over anything that matches *.yml . So while I work on a fix for this, could you just do inside your VOXL touch /data/modalai/something.yml. ? This should remove that error.

Keep me posted,
Thomas

Ajay Bidyarthy

@thomas
I attempted to use the voxl-logger by creating an empty something.yml file in the directory and executing the command. Despite encountering no errors during the execution, I did not receive any output beyond the line "connected to imu 0 sensor." My goal is to obtain benchmarking output similar to the tables shown in the ModalAI documentation: Benchmarks Documentation.

Could you kindly assist with this issue? I believe it would be more efficient if we could schedule a time for a discussion when you are available. This would allow us to address any questions or concerns promptly without the need for multiple back-and-forth messages.

Thank you for your time and assistance.

Ajay Bidyarthy

@thomas
Hi thomas,
Did you get a chance to check this issue? I am trying to benchmark the my models but I dont see any output or number I could use to summarize my benchmark or something.

@Ajay-Bidyarthy

Yeah so when you run voxl-logger with -s 5000 you're running it for that many samples and it isn't going to have any printouts. I'm not sure what tables you're talking about with respect to the benchmarks documentation, maybe you could give me a link?

If you just want to collect a voxl-logger log, your commands above should work fine. You can see that it's creating logs in the /data/voxl-logger/ directory which you can replay with the voxl-logger command.

I help with the forum in addition to my work and so I don't have time for calls. If you're interested in setting up a bigger support effort, you can send an email to sales@modalai.com .

Hope this helps!

Thomas

Ajay Bidyarthy

@thomas
Hi thomas,
Yes, so the table I'm referring to is under the benchmark section here: https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks
So the thing is when I run the voxl-looger with -s 5000 I also expect the code to run for that many samples and output some log, but as you see from my screenshot, it just outputs ... ' samples: 0'

And no worries thomas, I do appreciate your time and help here

Regards

@Ajay-Bidyarthy

Okay yeah let's back up here - why are we even trying to get a log with IMU samples in it? If you want to benchmark voxl-tflite-server, you don't need IMU samples to do so. What I had meant by all of this was: you don't have a camera attached to your VOXL if I remember correctly and so instead of attaching one, we could use a voxl-logger log which had camera data in it for replaying. This is because when you replay a voxl-logger log, it writes over all of the pipes. If we had a log which had camera data in it, you could replay it and your VOXL would "think" it had a camera attached and so then when you run voxl-tflite-server it could use the data coming from those pipes.

In this case, I probably need to provide you with a log that has camera data in it. No worries though, I have lots of logs and can upload one here. Do you have any preferences of what's in the log? I was just going to take my drone and wave it around for a few seconds but if you'd like something more we can do that. I'll also verify that this log can be used by tflite-server and what steps need to be taken.

If that seems reasonable let me know and I'll get started.

-Thomas

Ajay Bidyarthy

@thomas
Oh I understand now. So using your camera data voxl logs. We can simly use those logs(camera data) to benchmark tfliteserver and get the benchmark stats like shown here:
https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks

I would appreciate a lot if you could share them. Since it would be good to use those logs/same camera data to compare stats from diff custom tflite models

Thanks a lot

@Ajay-Bidyarthy

Yep exactly! Let me get working on a log right now for you to use and then I'll upload it here and give you instructions on how to use it.

Thomas

@Ajay-Bidyarthy

Here's a link to a .tar with a voxl-logger log that I recorded! https://drive.google.com/file/d/1c-snBLT6mmrwGpKE2WhVR-YQgumN0s6o/view?usp=sharing

You should use ADB to push this file onto VOXL and then use tar -xvf customlog.tar to unzip the file, it should unpack into /data/voxl-logger/log0009/. Then you should be able to voxl-replay -p /data/voxl-logger/log0009/. The only other important thing you need to do is edit voxl-tflite-server to predict from the tracking camera in /etc/modalai/voxl-tflite-server.conf.

Thomas

Ajay Bidyarthy

@thomas
Hey thomas,
So I have followed your steps and loaded the log0009 data and then ran voxl-replay command while running voxl-tflite-server correctly configured with tracking option. But nothing happens after that it seems.

So the output I'm expecting at the end is to be able to have something like
Model Task Avg Cpu Inference(ms) Avg Gpu Inference(ms) Avg NNAPI Inference(ms) Max Frames Per Second(fps) Input Dimensions Source
custom-tflite-model Object Detection 33.89ms 24.68ms 34.42ms 34.86750349 [1,300,300,3]

Where can I get such output after running the voxl-replay command. I think I'm missing some steps after that.

Please let me know what I'm missing here.
Thanks

@Ajay-Bidyarthy

Yeah, so my default all of our services don't print anything out when run. Do voxl-tflite-server -t for timing printouts.

This won't give you a super organized printout like what you are writing. But it will give you out data on the per-iteration callback times and the breakdown per function. You'll then need to modify /etc/modalai/voxl-tflite-server.conf in different ways to get different timing printouts in order to build up a table of benchmarks.

Thomas

Ajay Bidyarthy

@thomas
Okay, So i did run voxl-tflite-server -t alongside voxl-replay command but am still not getting anything.
For yolov5 I get no output after running voxl replay. it just outputs this:

For my custom yolov7 nd yolov8 I get these errors:

I even referred to this thread https://forum.modalai.com/topic/2997/can-anyone-recommend-a-tflite-colab-notebook-for-voxl2-training/3

and configured the main.cpp file by adding if else statement for custom model to use YOLO model instead of OBJECT_DETECTION as the default one before even running the above commands.

I'm not sure why am I still not getting any output.
Thanks

@Ajay-Bidyarthy I'll look into it today.

Thomas

@thomas @Ajay-Bidyarthy

Sadly I won't have time to look at this today and am out of office tomorrow so I can't fully debug this for you. Let's focus just on YOLOv5 right now since that's the one we have onboard by default and it isn't segfaulting. Make sure you're launching tflite-server in a terminal and then in a separate terminal running the voxl-replay command.

I apologize for this, sorry I didn't have time for it today.

Thomas