ModalAI Forum

@thomas
Hey thomas,
Hope you're doing well,
So I have a question - The three lines of results which we get after running the models :

1. Preprocessing time , 2. Inference Time, 3. Post processing Time
   Are they specific to CPU only. or are they according to if the task is delgated to GPU or something.

If its the later case, then can we manually run the tflite-server by manually delegating the model to the GPU or DPU, instead of it automatically detecting and processing accordingly.
Regards,
Bhavya

@thomas
Hi thomas,
So I was assuming the labels which are used for yolov5 which is present in the repo by default would work fine with custom models(yolov7 and yolov8) as well.

And yes the last screenshot is for yolov8.

@thomas
Hi thomas,
So we were able to run our custom tflite models using the updated tflite-server 0.3.3 . But now we are facing some new problem which is related to our custom tflite files. We are getting Segmentation fault errors as shown.


Could you help us regarding what exactly could we be missing here. Since I followed the steps mentioned in the documentation to create the compatible tflite models (using fp16)
https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks

Is there something we can test externally to check if our tflite model is compatible or not? Like some code or tool.

Regards,

@thomas
Hi thomas,
So I followed just the 4 steps you shared and its working perfectly fine with the default models.

But when i deploy my custom voxl-tflite-server deb package onto the chip it says warning downgrading from voxl-tflite-server 0.3.3 to 0.3.2. So maybe I feel the issue is with the deb package of voxl-tflite-server I'm building with my custom tflite models and building the package out of it and deploying.
Maybe you can update the gitlab repo so that I can build my custom voxl-tflite-server deb package and deploy it.
Let me know if you can do this.

Thanks
file:///home/bhavya/Pictures/Screenshots/Screenshot%20from%202024-04-17%2012-28-53.png

@thomas
Hi thomas,
Yes so right now we tried figuring out if maybe the issue is with the model not being specific to voxl2mini or maybe the logs are generated based on voxl2. But I dont think thats the case.
The thing which I'm concerned about is:

1. How do we identify or pinpoint the problem ...like are we supposed to see the output after the 'Disconnected from camera server' line or before? And if we are supposed to see it and are not able to... then where exactly is the problem happening in the src/main.cpp file of voxl-tflite-server. Can we debug that somehow?
2. Also did you check running the voxl-replay logger command directly without having any camera setup or camera-server configured.... similar to the way we are currently running on our end.

@thomas
Hey thomas,
So I tried with this new sdk. Unfortunately, I still dont see any output here as well

@thomas
Also, Now that I have flashed the SDK onto my chip its showing voxl2 in the shell screen instead of voxl2-mini. And even if I try and go ahead and do the following steps. I still cant get the desired output.

file:///home/bhavya/Pictures/Screenshots/Screenshot%20from%202024-04-15%2014-29-02.png

@thomas
Because I think this SDK is meant for M-0054 while our version is M0104. Let me know if we can go ahead and flash this SDK or not?
Thanks

@thomas
Hi thomas,
Is this package for voxl2min or voxl2?
Regards

@thomas
Hi thomas,
Could you please provide us the latest nightly sdk build for VOXL2 mini if you think we can work with that for now?

Regards,
OKSI Team

@thomas
Hi,
Thanks for this info. So I have tried accessing the portal by running the voxl-my-ip command but found nothing getting printed out when I run that command. I inspected to make sure my voxl-portal service is running perfectly but still cant seem to get the ip for portal.
Also, can you push the latest SDK as well or share the deb with me.
file:///home/bhavya/Downloads/WhatsApp%20Image%202024-04-09%20at%2010.46.27%20AM.jpeg

Thanks

@thomas
No worries. Ok, but even for yolov5 I've been running both voxl replay and tflite server commands in separate windows/terminals but still facing the issue.

Let me know why is it so and how can we resolve this.
Thanks

@thomas
Okay, So i did run voxl-tflite-server -t alongside voxl-replay command but am still not getting anything.
For yolov5 I get no output after running voxl replay. it just outputs this:

For my custom yolov7 nd yolov8 I get these errors:

I even referred to this thread https://forum.modalai.com/topic/2997/can-anyone-recommend-a-tflite-colab-notebook-for-voxl2-training/3

and configured the main.cpp file by adding if else statement for custom model to use YOLO model instead of OBJECT_DETECTION as the default one before even running the above commands.

I'm not sure why am I still not getting any output.
Thanks

@thomas
Hey thomas,
So I have followed your steps and loaded the log0009 data and then ran voxl-replay command while running voxl-tflite-server correctly configured with tracking option. But nothing happens after that it seems.

So the output I'm expecting at the end is to be able to have something like
Model Task Avg Cpu Inference(ms) Avg Gpu Inference(ms) Avg NNAPI Inference(ms) Max Frames Per Second(fps) Input Dimensions Source
custom-tflite-model Object Detection 33.89ms 24.68ms 34.42ms 34.86750349 [1,300,300,3]

Where can I get such output after running the voxl-replay command. I think I'm missing some steps after that.

Please let me know what I'm missing here.
Thanks

@thomas
Oh I understand now. So using your camera data voxl logs. We can simly use those logs(camera data) to benchmark tfliteserver and get the benchmark stats like shown here:
https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks

I would appreciate a lot if you could share them. Since it would be good to use those logs/same camera data to compare stats from diff custom tflite models

Thanks a lot

@thomas
Hi thomas,
Yes, so the table I'm referring to is under the benchmark section here: https://docs.modalai.com/voxl-tflite-server-0_9/#benchmarks
So the thing is when I run the voxl-looger with -s 5000 I also expect the code to run for that many samples and output some log, but as you see from my screenshot, it just outputs ... ' samples: 0'

And no worries thomas, I do appreciate your time and help here

Regards

@thomas
Hi thomas,
Did you get a chance to check this issue? I am trying to benchmark the my models but I dont see any output or number I could use to summarize my benchmark or something.

@thomas
I attempted to use the voxl-logger by creating an empty something.yml file in the directory and executing the command. Despite encountering no errors during the execution, I did not receive any output beyond the line "connected to imu 0 sensor." My goal is to obtain benchmarking output similar to the tables shown in the ModalAI documentation: Benchmarks Documentation.

Could you kindly assist with this issue? I believe it would be more efficient if we could schedule a time for a discussion when you are available. This would allow us to address any questions or concerns promptly without the need for multiple back-and-forth messages.

Thank you for your time and assistance.

@thomas
Hi thomas,
So I did try calibrating the imu sensor using voxl-calibrate-imu. But I still dont see the .yml file in the data/modalai directory.
So I'll just share the brief overview on what steps I followed to get to the current point:

1. I connected VOXL (with only the default sensors, I did not attach any sensor seperately) to my computer.
2. I created a deb package to deploy to VOXL by loading my custom tflite files in the misc_files/usr/bin/dnn directory of the voxl-tflite-server with correspodning changs in the voxl-configure-tflite file present in the scripts/ qrb5165 folder. And then following the build and deploy commands to deploy the package to voxl
3. I then did adb shell to load into my VOXL and do see all the tflite files present there. I then ran voxl-configure-tflite command to load my custom tflite model.
4. Following that I simply executed voxl-tflite-server command
5. In another similar terminal I then tried benchmarking by doing voxl-logger command as shown and am now getting the error.
   file:///home/bhavya/Downloads/screen1.jpeg

file:///home/bhavya/Downloads/screen2.jpeg

Thanks

@thomas
Yes, so we have the directory present, but it only has the sku.txt file there. I think its because of some .yml file being missing we get error. So what should we do to load that file successfully in the dir