# voxl-tflite-server not working on latest SW Source: https://forum.modalai.com/topic/528/voxl-tflite-server-not-working-on-latest-sw Category: General Questions (https://forum.modalai.com/category/2/general-questions) Posted: 2021-10-28 23:05:14 UTC by jaredjohansen Replies: 5 · Views: 1322 ## jaredjohansen · 2021-10-28 23:05:14 UTC I am starting my VOXL with the latest code. From [here](https://developer.modalai.com/asset), I download and extract voxl_platform_3-3-0-0.4.6-b.tar.gz. I follow [these steps](https://docs.modalai.com/flash-system-image/) to flash the system image (and elect to wipe the `/data` partition). When that completes, I connect to wifi and install the voxl-suite. When that completes, I run `voxl-configure-mpa --wizard` to configure the services. Among other things, I select hires + tracking for the `voxl-camera-server` and enable the `voxl-tflite-server`. When that completes, I install voxl-portal (`opkg install voxl-portal`), enable that service (`systemctl enable voxl-portal`), and set the VOXL into softap mode. I now hard reboot the VOXL. Once it reboots, I can see the hires, tracking, and qvio-overlay images in the voxl-portal on my laptop. I cannot see the tflite-overlay. `voxl-inspect-services` shows that it is enabled, but not running. I run `journalctl -u voxl-tflite-server.service` and see the following messages: ``` -- Logs begin at Thu 1970-01-01 00:11:17 UTC, end at Thu 1970-01-01 00:13:33 UTC. -- Jan 01 00:11:20 apq8096 systemd[1]: Started voxl-tflite-server. Jan 01 00:11:20 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Main process exited, code=exited, status=127/n/a[[0m Jan 01 00:11:20 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Unit entered failed state.[[0m Jan 01 00:11:20 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Failed with result 'exit-code'.[[0m Jan 01 00:11:21 apq8096 systemd[1]: voxl-tflite-server.service: Service hold-off time over, scheduling restart. Jan 01 00:11:21 apq8096 systemd[1]: Stopped voxl-tflite-server. Jan 01 00:11:21 apq8096 systemd[1]: Started voxl-tflite-server. Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Main process exited, code=exited, status=127/n/a[[0m Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Unit entered failed state.[[0m Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Failed with result 'exit-code'.[[0m Jan 01 00:11:21 apq8096 systemd[1]: voxl-tflite-server.service: Service hold-off time over, scheduling restart. Jan 01 00:11:21 apq8096 systemd[1]: Stopped voxl-tflite-server. Jan 01 00:11:21 apq8096 systemd[1]: Started voxl-tflite-server. Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Main process exited, code=exited, status=127/n/a[[0m Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Unit entered failed state.[[0m Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Failed with result 'exit-code'.[[0m Jan 01 00:11:21 apq8096 systemd[1]: voxl-tflite-server.service: Service hold-off time over, scheduling restart. Jan 01 00:11:21 apq8096 systemd[1]: Stopped voxl-tflite-server. Jan 01 00:11:21 apq8096 systemd[1]: Started voxl-tflite-server. Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Main process exited, code=exited, status=127/n/a[[0m Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Unit entered failed state.[[0m Jan 01 00:11:21 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Failed with result 'exit-code'.[[0m Jan 01 00:11:22 apq8096 systemd[1]: voxl-tflite-server.service: Service hold-off time over, scheduling restart. Jan 01 00:11:22 apq8096 systemd[1]: Stopped voxl-tflite-server. Jan 01 00:11:22 apq8096 systemd[1]: Started voxl-tflite-server. Jan 01 00:11:22 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Main process exited, code=exited, status=127/n/a[[0m Jan 01 00:11:22 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Unit entered failed state.[[0m Jan 01 00:11:22 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Failed with result 'exit-code'.[[0m Jan 01 00:11:22 apq8096 systemd[1]: voxl-tflite-server.service: Service hold-off time over, scheduling restart. Jan 01 00:11:22 apq8096 systemd[1]: Stopped voxl-tflite-server. Jan 01 00:11:22 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Start request repeated too quickly.[[0m Jan 01 00:11:22 apq8096 systemd[1]: [[1;31mFailed to start voxl-tflite-server.[[0m Jan 01 00:11:22 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Unit entered failed state.[[0m Jan 01 00:11:22 apq8096 systemd[1]: [[1;39mvoxl-tflite-server.service: Failed with result 'start-limit'.[[0m Jan 01 00:11:22 apq8096 bash[2146]: /usr/bin/voxl-tflite-server: error while loading shared libraries: libEGL_adreno.so: cannot o pen shared object file: No such file or directory Jan 01 00:11:22 apq8096 bash[2351]: /usr/bin/voxl-tflite-server: error while loading shared libraries: libEGL_adreno.so: cannot o pen shared object file: No such file or directory Jan 01 00:11:22 apq8096 bash[2389]: /usr/bin/voxl-tflite-server: error while loading shared libraries: libEGL_adreno.so: cannot o pen shared object file: No such file or directory Jan 01 00:11:22 apq8096 bash[2583]: /usr/bin/voxl-tflite-server: error while loading shared libraries: libEGL_adreno.so: cannot o pen shared object file: No such file or directory Jan 01 00:11:22 apq8096 bash[2683]: /usr/bin/voxl-tflite-server: error while loading shared libraries: libEGL_adreno.so: cannot o pen shared object file: No such file or directory ``` I found that libEGL_adreno.so exists in the filesystem: ``` yocto:~$ ls -al /share/qualcomm-proprietary/usr/lib64/libEGL_adreno.so -rw-r--r--. 1 root root 119624 May 20 2021 /share/qualcomm-proprietary/usr/lib64/libEGL_adreno.so ``` I tried adding that path to `LD_LIBRARY_PATH` and restarting the `voxl-tflite-service`, but it results in the same error. Can you please help me know what I need to do to get this working? ## Reply by Guest · 2021-10-29 16:49:14 UTC Hi jared, This is a strange bug, as voxl-tflite-server is expecting those libraries to be under /usr/lib64/. I am looking into this now, but a quick fix would be manually copying those files into the lib64/ directory using this command: ``` cp -r /share/qualcomm-proprietary/usr/lib64/* /usr/lib64/ ``` ## Reply by jaredjohansen · 2021-10-29 18:45:40 UTC @Matt-Turi, that got me a step further! After running the command you specified, I ran `systemctl restart voxl-tflite-server`. Now the service runs (without dying), but no data comes out. This is the output of `journalctl -u voxl-tflite-server.service` (immediately after I restarted the service): ``` Jan 01 00:31:11 apq8096 systemd[1]: Stopped voxl-tflite-server. Jan 01 00:31:11 apq8096 systemd[1]: Started voxl-tflite-server. Jan 01 00:31:11 apq8096 bash[3578]: Warning: please export TSAN_OPTIONS='ignore_noninstrumented_modules=1' to avoid false positive reports from the OpenMP runtime.! Jan 01 00:31:11 apq8096 bash[3578]: Creating new config file: /etc/modalai/voxl-tflite-server.conf Jan 01 00:31:11 apq8096 bash[3578]: ------VOXL TFLite Server------ Jan 01 00:31:11 apq8096 bash[3578]: INFO: Created TensorFlow Lite delegate for GPU. Jan 01 00:31:11 apq8096 bash[3578]: GPU acceleration is SUPPORTED on this platform Jan 01 00:31:11 apq8096 bash[3578]: ERROR: Next operations are not supported by GPU delegate: Jan 01 00:31:11 apq8096 bash[3578]: CUSTOM TFLite_Detection_PostProcess: Operation is not supported. Jan 01 00:31:11 apq8096 bash[3578]: First 114 operations will run on the GPU, and the remaining 1 on the CPU. Jan 01 00:31:14 apq8096 bash[3578]: INFO: Initialized OpenCL-based API. ``` If I reboot, I get the same (new) behavior. If I run `voxl-tflite-server -d`, I get the following output (and then it hangs): ``` yocto:/$ voxl-tflite-server -d Enabling debug mode exising instance of voxl-tflite-server found, attempting to stop it ================================================================= skip_n_frames: 5 ================================================================= ================================================================= model: /usr/bin/dnn/ssdlite_mobilenet_v2_coco.tflite ================================================================= ================================================================= input_pipe: /run/mpa/hires_preview/ ================================================================= ------VOXL TFLite Server------ Loaded model /usr/bin/dnn/ssdlite_mobilenet_v2_coco.tflite Resolved reporter INFO: Created TensorFlow Lite delegate for GPU. GPU acceleration is SUPPORTED on this platform ERROR: Next operations are not supported by GPU delegate: CUSTOM TFLite_Detection_PostProcess: Operation is not supported. First 114 operations will run on the GPU, and the remaining 1 on the CPU. INFO: Initialized OpenCL-based API. Applied delegate ------Setting TFLiteThread to ready!! W: 300 H: 300 C:3 ``` That command shows that it is looking for this input_pipe: `/run/mpa/hires_preview/`. I looked in that directory and that pipe does not exist (although `/run/mpa/hires/` does). I edited `/etc/modalai/voxl-tflite-server.conf`, changed the input_pipe to `/run/mpa/hires/`, and restarted the `voxl-tflite-server`. That didn't work. I presume `voxl-tflite-server` is not working because there is no `/run/mpa/hires_preview/`. Is that true? If so, any idea why there is no `/run/mpa/hires_preview/`? ## Reply by jaredjohansen · 2021-10-29 20:32:11 UTC I ran those commands: ``` yocto:/$ opkg update Downloading http://voxl-packages.modalai.com/stable/Packages.gz. Updated source 'stable'. yocto:/$ opkg upgrade Not selecting librc_math 1.1.2 as installing it would break existing dependencies. Not selecting opencv 4.5.1 as installing it would break existing dependencies. Not selecting libmodal_pipe 1.2.2 as installing it would break existing dependencies. Upgrading voxl-utils from 0.8.1 to 0.8.2 on root. Downloading http://voxl-packages.modalai.com/stable/voxl-utils_0.8.2.ipk. Configuring voxl-utils. done installing voxl-utils ``` After running this, I hard rebooted my VOXL. Same behavior as before. In regards to `opkg upgrade`, I'm not sure why it wants libmodal_pipe 1.2.2 since there is a 2.1.1 version. Here's the result of `opkg info libmodal_pipe`. It looks like there are two versions installed? ``` yocto:/$ opkg info libmodal_pipe Package: libmodal_pipe Version: 2.1.1 Depends: libmodal_json Conflicts: libvoxl_pipe, modal_pipe_interfaces Status: install ok installed Section: base Architecture: all Maintainer: james@modalai.com MD5Sum: 8b7f79e8ef7db663361bc2ce98ae6197 Size: 110208 Filename: libmodal_pipe_2.1.1.ipk Description: Interface for services to communicate over posix named pipes Installed-Time: 1635461181 Package: libmodal_pipe Version: 1.2.2 Status: unknown ok not-installed Section: base Architecture: armv7a Maintainer: james@modalai.com MD5Sum: ebfe02800708c1813be3bf313aaa23c0 Size: 27426 Filename: libmodal_pipe_1.2.2.ipk Description: Interface for services to communicate over posix named pipes ``` ## Reply by Guest · 2021-10-29 21:04:17 UTC Looks like it was a bug with some of our recent package updates. The latest voxl-tflite-server (v0.1.6) and voxl-utils (v0.8.2) will fix these issues, and can be obtained from the opkg dev or stable repo. Once installed, you can edit the config file and choose either "hires" (not hires_preview) or "tracking", but the default will be hires. ## Reply by jaredjohansen · 2021-10-29 21:24:17 UTC I already had voxl-utils v0.8.2 installed (see earlier post), so I did not update that. I downloaded voxl-tflite-server v0.1.6 from http://voxl-packages.modalai.com/dev/ and adb push'ed it to my voxl. I then installed it (`opkg install ipk/voxl-tflite-server_0.1.6_202110292056.ipk`) and rebooted my voxl. I saw the same behavior as before until I modified `/etc/modalai/voxl-tflite-server.conf`. Once I set ``` "input_pipe": "/run/mpa/hires/" ``` and restarted the `voxl-tflite-server`, it began to work! Thanks, @Matt-Turi, for helping me resolve this issue!