camera_image_metadata_t framerate is zero from lepton0_color
-
Hello,
I have an image processing application that is getting camera frames from the lepton0_color pipe. However, in the camera_image_metadata_t the framerate is zero. Is this expected or do you have any suggestions on how to fix it? For reference, when my application receives frames from the hires_down_small_color pipe the metadata has the framerate populated correctly.Below is a snippet of the logs from my application showing the framerate of zero.
_camera_helper_cbWARNING: Camera frame rate of 0 Hz is < required frame rate of 1 Hz. Camera Image Metadata: Magic Number: 0x564f584c Timestamp (ns): 72864071544650 Frame ID: 632968 Dimensions: 160x120 pixels Size (bytes): 57600 Stride: 480 bytes/row Exposure (ns): 0 Gain (ISO): 0 Format: 10 Framerate (Hz): 0 Reserved: 0I am on a Starling 2 Max. Below is my current configuration.
voxl2:~$ voxl-version -------------------------------------------------------------------------------- system-image: 1.8.02-M0054-14.1a-perf kernel: #1 SMP PREEMPT Mon Nov 11 22:59:41 UTC 2024 4.19.125 -------------------------------------------------------------------------------- hw platform: M0054 mach.var: 1.2.1 -------------------------------------------------------------------------------- voxl-suite: 1.4.1 -------------------------------------------------------------------------------- Packages: Repo: http://voxl-packages.modalai.com/ ./dists/qrb5165/sdk-1.4/binary-arm64/ Last Updated: 2023-03-02 12:58:40 WARNING: repo file has changed since last update, packages may have originated from a different repo List: kernel-module-voxl-fsync-mod-4.19.125 1.0-r0 kernel-module-voxl-gpio-mod-4.19.125 1.0-r0 kernel-module-voxl-platform-mod-4.19.125 1.0-r0 libfc-sensor 1.0.7 libmodal-cv 0.5.16 libmodal-exposure 0.1.3 libmodal-journal 0.2.2 libmodal-json 0.4.3 libmodal-pipe 2.10.6 libqrb5165-io 0.4.9 libvoxl-cci-direct 0.2.5 libvoxl-cutils 0.1.1 modalai-slpi 1.1.19 mv-voxl 0.1-r0 qrb5165-bind 0.1-r0 qrb5165-dfs-server 0.2.0 qrb5165-imu-server 1.1.2 qrb5165-rangefinder-server 0.1.4 qrb5165-slpi-test-sig 01-r0 qrb5165-system-tweaks 0.3.4 qrb5165-tflite 2.8.0-2 voxl-bind-spektrum 0.1.1 voxl-camera-calibration 0.5.9 voxl-camera-server 2.1.1 voxl-ceres-solver 2:1.14.0-10 voxl-configurator 0.9.7 voxl-cpu-monitor 0.5.3 voxl-docker-support 1.3.1 voxl-elrs 0.4.1 voxl-esc 1.5.1 voxl-feature-tracker 0.5.2 voxl-flow-server 0.3.6 voxl-fsync-mod 1.0-r0 voxl-gphoto2-server 0.0.10 voxl-gpio-mod 1.0-r0 voxl-io-server 0.0.4 voxl-jpeg-turbo 2.1.3-5 voxl-lepton-server 1.3.3 voxl-lepton-tracker 0.0.4 voxl-libgphoto2 0.0.4 voxl-libuvc 1.0.7 voxl-logger 0.4.9 voxl-mavcam-manager 0.5.7 voxl-mavlink 0.1.1 voxl-mavlink-server 1.4.4 voxl-modem 1.1.5 voxl-mongoose 7.7.0-1 voxl-mpa-to-ros 0.3.9 voxl-mpa-tools 1.3.7 voxl-open-vins 0.4.16 voxl-open-vins-server 0.3.0 voxl-opencv 4.5.5-2 voxl-osd 0.1.1 voxl-platform-mod 1.0-r0 voxl-portal 0.7.5 voxl-px4 1.14.0-2.0.94 voxl-px4-imu-server 0.1.2 voxl-px4-params 0.6.3 voxl-qvio-server 1.1.1 voxl-remote-id 0.0.9 voxl-reset-slpi 0.0.1 voxl-state-estimator 0.0.4 voxl-streamer 0.7.5 voxl-suite 1.4.1 voxl-tag-detector 0.0.4 voxl-tflite-server 0.3.9 voxl-utils 1.4.4 voxl-uvc-server 0.1.7 voxl-vision-hub 1.8.17 voxl-vtx 1.1.8 voxl2-io 0.0.3 voxl2-system-image 1.8.02-r0 voxl2-wlan 1.0-r0 -------------------------------------------------------------------------------- -
@paul-foley , it looks like our
voxl-lepton-serveris not setting the FPS field in the metadata : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-lepton-server/-/blob/master/src/publisher.c?ref_type=heads#L338 . I do see above that we do set the framerate in json info about the output pipe (cJSON_AddNumberToObject(json, "framerate", 9);) and the framerate is hardcoded to 9, since i guess all lepton sensors are running at 9fps.I just made the change to publish the fps : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-lepton-server/-/commit/110b9cce4c021322566e46f36d6878fca55f7d26
You are using an older voxl SDK, so installing latest nightly package (from http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64/ will probably not work). So you probably need to take the source of the version you are running now (1.3.3 : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-lepton-server/-/tree/v1.3.3?ref_type=tags) and apply the same change in the source code and build it.
Since i just pushed to dev branch, a new version of leptop server should appear in our package repo tomorrow, and you can download and try it. it might work, but your SDK is 2 major versions down from where we are now (1.4.x vs 1.6.x), so you may want to just build the patched version 1.3.3 :
git clone https://gitlab.com/voxl-public/voxl-sdk/services/voxl-lepton-server.git cd voxl-lepton-server git checkout v1.3.3 #make the change in publisher.c to include fps ... ... #start voxl-cross docker (you can download it in our downloads section: https://developer.modalai.com/asset) docker run -it --rm -v `pwd`:/opt/code -w /opt/code voxl-cross:V4.6 bash ./install_build_deps.sh qrb5165 sdk-1.4 ./build.sh qrb5165 ./make_package.sh #then, outside docker, deploy the new package to voxl2 via adb ./deploy_to_voxl.shPlease let me know if this works for you!
Alex
