M0040 Time of Flight sensor FPS
-
@Darshit-Desai No, there is not currently a way to increase to 30FPS. We will take it into consideration for the future
@Moderator Is it because of the CPU computation constraints or is it because of the sensor itself? If it's because of the sensor, will upgrading my current ToF to the new next-gen ToF sensor make a difference?
-
@Moderator Is it because of the CPU computation constraints or is it because of the sensor itself? If it's because of the sensor, will upgrading my current ToF to the new next-gen ToF sensor make a difference?
@Darshit-Desai , the limitation is coming from the current camera server configuration, not from the sensor itself. I discussed this with the team and we will try to add more mode options for TOF sensors in short term. We need to do a bit more investigation before we can provide ETA. The change should be relatively small, hopefully will have good news later this week.
Alex
-
@Darshit-Desai , the limitation is coming from the current camera server configuration, not from the sensor itself. I discussed this with the team and we will try to add more mode options for TOF sensors in short term. We need to do a bit more investigation before we can provide ETA. The change should be relatively small, hopefully will have good news later this week.
Alex
@Alex-Kushleyev Wow, thats really good news.
I was just starting to test a constant velocity interpolation model (Kalman filter type) for increasing the frame rate of the ToF and hopefully match it with the Neural net Detection frame rate for my tracking task. (Artificially and highly inaccurate for non linear drone movements). This natural increase in frame rate from the sensor itself would be much better for my project
-
@Alex-Kushleyev Wow, thats really good news.
I was just starting to test a constant velocity interpolation model (Kalman filter type) for increasing the frame rate of the ToF and hopefully match it with the Neural net Detection frame rate for my tracking task. (Artificially and highly inaccurate for non linear drone movements). This natural increase in frame rate from the sensor itself would be much better for my project
@Darshit-Desai , can you please let me know which version of
voxl-camera-serveryou are using now? -
@Darshit-Desai , can you please let me know which version of
voxl-camera-serveryou are using now?@Alex-Kushleyev Is the command to check the voxl-camera-server voxl-camera-server --version?
Just wanted to inform you that I am using VOXL SDK 1.1.2, which is the latest release on this website. Probably the version would be the one coupled with the latest SDK update. I haven't made any changes to the voxl-camera-server on top of the updated SDK.
-
@Alex-Kushleyev Is the command to check the voxl-camera-server voxl-camera-server --version?
Just wanted to inform you that I am using VOXL SDK 1.1.2, which is the latest release on this website. Probably the version would be the one coupled with the latest SDK update. I haven't made any changes to the voxl-camera-server on top of the updated SDK.
@Darshit-Desai you can see by running voxl-version , it will print versions of all voxl packages
-
@Darshit-Desai you can see by running voxl-version , it will print versions of all voxl packages
@Alex-Kushleyev Here's the version details of all packages, the camera-server version is 1.8.9
-
@Alex-Kushleyev Here's the version details of all packages, the camera-server version is 1.8.9
@Darshit-Desai OK, great so the version is 1.8.9 which came from skd-1.1.2.
I will experiment using that version so that you can test it directly. The latest
devversion of voxl-camera-server on gitlab has some updates for the new TOF sensor and it's not yet officially "released". -
@Darshit-Desai OK, great so the version is 1.8.9 which came from skd-1.1.2.
I will experiment using that version so that you can test it directly. The latest
devversion of voxl-camera-server on gitlab has some updates for the new TOF sensor and it's not yet officially "released".@Alex-Kushleyev Thank you, but if I build and deploy the
devbranch code, will that show the same error as shown in this post,I was trying to find that error message in the code repo of
voxl-camera-serverbut I didn't find this specific error line,@Darshit-Desai said in M0040 Time of Flight sensor FPS:
@Moderator I found a way to increase FPS from older posts, Is there a way to increase FPS more then 15 for the ToF camera as in the data sheet it still says the FPS can go upto 45 Hz. I want to increase till atleast 25 Hz and if the CPU still has some power left upto 30 Hz. But when i do that the camera server has an error saying it can either take 5 or 15 FPS.
-
@Alex-Kushleyev Thank you, but if I build and deploy the
devbranch code, will that show the same error as shown in this post,I was trying to find that error message in the code repo of
voxl-camera-serverbut I didn't find this specific error line,@Darshit-Desai said in M0040 Time of Flight sensor FPS:
@Moderator I found a way to increase FPS from older posts, Is there a way to increase FPS more then 15 for the ToF camera as in the data sheet it still says the FPS can go upto 45 Hz. I want to increase till atleast 25 Hz and if the CPU still has some power left upto 30 Hz. But when i do that the camera server has an error saying it can either take 5 or 15 FPS.
@Darshit-Desai the error is here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp#L561
But you can't just modify the check condition. In order to switch to a higher FPS than 15Hz, the sensor needs to use a medium or short range mode (currently long range mode). So a few other things need to change.
So do you want me to use dev version or sdk 1.1.2 version as starting point? It may be easier for you, since everything will remain the same (no other changes in camera server).
this change will (eventually) make it to dev anyway, i could go directly to dev (in fact it will be simpler for me), but i am offering you a choice
-
@Darshit-Desai the error is here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/dev/src/hal3_camera_mgr.cpp#L561
But you can't just modify the check condition. In order to switch to a higher FPS than 15Hz, the sensor needs to use a medium or short range mode (currently long range mode). So a few other things need to change.
So do you want me to use dev version or sdk 1.1.2 version as starting point? It may be easier for you, since everything will remain the same (no other changes in camera server).
this change will (eventually) make it to dev anyway, i could go directly to dev (in fact it will be simpler for me), but i am offering you a choice
@Alex-Kushleyev Presently I only require a bump in the FPS, so I suggest you do changes by taking sdk1.1.2 as the base. If you use the
devversion there might be unaccounted bugs which I don't want in my build -
@Alex-Kushleyev Presently I only require a bump in the FPS, so I suggest you do changes by taking sdk1.1.2 as the base. If you use the
devversion there might be unaccounted bugs which I don't want in my build@Darshit-Desai , yes that is exactly my thought. i will use sdk1.1.2 as the base.
-
@Darshit-Desai , yes that is exactly my thought. i will use sdk1.1.2 as the base.
@Darshit-Desai , to my own surprise, this was quick. I was able to test at 30, 45 and 60fps all of which will use the short range mode of the sensor. I have done minimal testing, but the depth, confidence and Ir images look good in
voxl-portal
Files · sdk1.1.2-add-tof-fps-options · voxl / VOXL SDK / Services / voxl-camera-server · GitLab
GitLab.com
GitLab (gitlab.com)
Just build voxl camera server from this branch, make package, install package on voxl2 and update your fps parameter from 15 to either 30, 45, 60 in
voxl-camera-server.conf
Let me know how it goes!
-
@Darshit-Desai , to my own surprise, this was quick. I was able to test at 30, 45 and 60fps all of which will use the short range mode of the sensor. I have done minimal testing, but the depth, confidence and Ir images look good in
voxl-portal
Files · sdk1.1.2-add-tof-fps-options · voxl / VOXL SDK / Services / voxl-camera-server · GitLab
GitLab.com
GitLab (gitlab.com)
Just build voxl camera server from this branch, make package, install package on voxl2 and update your fps parameter from 15 to either 30, 45, 60 in
voxl-camera-server.confLet me know how it goes!
@Alex-Kushleyev Thank you I am trying this out right now, Which option should I choose here for repo version
Also when I do git branch after cloning the repo it only shows
masterbranch I had to manually write the branch name -
@Alex-Kushleyev Thank you I am trying this out right now, Which option should I choose here for repo version
Also when I do git branch after cloning the repo it only shows
masterbranch I had to manually write the branch namegit branch -ashows all the branchesfor installing deps, use
sdk-1.1since we based this branch off sdk-1.1.2Alex
-
git branch -ashows all the branchesfor installing deps, use
sdk-1.1since we based this branch off sdk-1.1.2Alex
@Alex-Kushleyev Thanks it worked it shows point cloud at 30 FPS when I set it to 30 in .conf file
I will try it out with other services like mpatoros2, tfliteserver in tandem see if there is a drop in frame rate or performance
-
@Alex-Kushleyev Thanks it worked it shows point cloud at 30 FPS when I set it to 30 in .conf file
I will try it out with other services like mpatoros2, tfliteserver in tandem see if there is a drop in frame rate or performance
@Alex-Kushleyev right now the FPS values need to be a multiple of 15 for some reason because I tried 35, and 20 FPS and the camera server refuses to start
-
@Alex-Kushleyev right now the FPS values need to be a multiple of 15 for some reason because I tried 35, and 20 FPS and the camera server refuses to start
@Darshit-Desai , i will double check the available FPS. Not all arbitrary FPS are possible.
-
@Darshit-Desai , i will double check the available FPS. Not all arbitrary FPS are possible.
If you run “voxl-camera-server -d 0”, you will see a lot of debug messages including all possible tof modes. If you see something you want to try, you can modify the code. Mode9 is long range and mode5 is short range.
-
If you run “voxl-camera-server -d 0”, you will see a lot of debug messages including all possible tof modes. If you see something you want to try, you can modify the code. Mode9 is long range and mode5 is short range.
@Alex-Kushleyev I dont see mode descriptions when I try this
voxl2:/etc/modalai$ voxl-camera-server -d 0 detected system image 1.7.1 using new imx214 defaults ================================================================= configuration for 3 cameras: cam #0 name: tof sensor type: pmd-tof isEnabled: 1 camId: 0 camId2: -1 fps: 20 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 224 pre_height: 1557 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 ae_mode: off standby_enabled: 0 decimator: 5 independent_exposure:0 cam #1 name: hires sensor type: imx214 isEnabled: 1 camId: 1 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 0 pre_width: 640 pre_height: 480 en_raw_preview: 0 en_small_video: 1 small_video_width: 1024 small_video_height: 768 en_large_video: 1 large_video_width: 4208 large_video_height: 3120 en_snapshot: 1 snap_width: 4208 snap_height: 3120 ae_mode: isp standby_enabled: 0 decimator: 1 independent_exposure:0 cam #2 name: tracking sensor type: ov7251 isEnabled: 1 camId: 2 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 640 pre_height: 480 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 ae_mode: lme_msv standby_enabled: 0 decimator: 1 independent_exposure:0 ================================================================= DEBUG: Attempting to open the hal module DEBUG: SUCCESS: Camera module opened on attempt 0 DEBUG: ----------- Number of cameras: 3 DEBUG: Cam idx: 0, Cam slot: 0, Slave Address: 0x007A, Sensor Id: 0x003D DEBUG: Cam idx: 1, Cam slot: 2, Slave Address: 0x0020, Sensor Id: 0x0214 DEBUG: Cam idx: 2, Cam slot: 3, Slave Address: 0x00E2, Sensor Id: 0x7750 GPS server Connected DEBUG: Connected to cpu-monitor DEBUG: ------ voxl-camera-server: Starting 3 cameras Starting Camera: tof (id #0) DEBUG: Checking Gain limits for Camera: tof Using gain limits min: 54 max: 8000 DEBUG: cam ID 0 checking for fmt: 38 w: 224 h: 1557 o: 0 DEBUG: i: 0 fmt: 34 w: 176 h: 144 o:0 DEBUG: i: 4 fmt: 34 w: 176 h: 144 o:1 DEBUG: i: 8 fmt: 35 w: 176 h: 144 o:0 DEBUG: i: 12 fmt: 35 w: 176 h: 144 o:1 DEBUG: i: 16 fmt: 33 w: 176 h: 144 o:0 DEBUG: i: 20 fmt: 37 w: 224 h:1557 o:0 DEBUG: i: 24 fmt: 38 w: 224 h:1557 o:0 DEBUG: i: 28 fmt: 32 w: 224 h:1557 o:0 DEBUG: i: 32 fmt: 36 w: 224 h:1557 o:0 VERBOSE: Successfully found configuration match for camera 0: 224x1557 VERBOSE: Adding preview stream for camera: 0 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI VERBOSE: Opened GBM fd gbm_create_device(156): Info: backend name is: msm_drm VERBOSE: Created GBM device DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Converted gralloc flags 0x20900 to GBM flags 0x1400000 VERBOSE: Dumping GBM flags DEBUG: Found flag GBM_BO_USAGE_CAMERA_WRITE_QTI DEBUG: Found flag GBM_BO_USAGE_HW_COMPOSER_QTI DEBUG: Allocated BO with width=224 height=1557 stride=336 aligned_w=336 aligned_h=1557 size=524288 flags=0x20900 format=GBM_FORMAT_RAW12 DEBUG: Successfully set up pipeline for stream: PREVIEW VERBOSE: Entered thread: cam0-request(tid: 3925) DEBUG: Started Camera: tof Starting Camera: hires (id #1) ERROR: Invalid TOF framerate: 20, must be either 5 or 15 ERROR: Failed to construct request settings for camera: tof WARNING: Thread: tof request thread Received ESTOP VERBOSE: Leaving tof request thr ```ead
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login