TOF sensor not working with latest VOXL SW
-
I recently worked through an issue with @Matt-Turi where the voxl-tflite-server was not not working with the latest VOXL SW.
In that forum post, I was working with the hires + tracking cameras. With voxl-portal, I was able to see the hires, tflite-overlay, tracking, and qvio-overlay images.
Since I have a TOF sensor, I want to enable and use that sensor as well. Note that a week ago (with older VOXL SW) the VOXL I have was working properly with this sensor.
I ran
voxl-configure-camerasto change the configuration to option 6:Hires + TOF + Tracking. After doing that, thevoxl-camera-servercrashes. If I restart it, it dies again. If I change the camera configuration to hires, tracking, or hires+tracking, the voxl-camera-server works properly. Something seems amiss with the TOF sensor.I ran
journalctl -u voxl-camera-serverand got this output:Jan 01 00:00:07 apq8096 systemd[1]: Started voxl-camera-server. Jan 01 00:02:25 apq8096 bash[2130]: ------ voxl-camera-server INFO: Camera server exited gracefully Jan 01 00:02:25 apq8096 systemd[1]: Stopped voxl-camera-server. Jan 01 00:02:25 apq8096 systemd[1]: Started voxl-camera-server. Jan 01 00:02:26 apq8096 bash[3798]: ------ voxl-camera-server INFO: Camera server is now stopping Jan 01 00:02:26 apq8096 bash[3798]: There is a chance that it may segfault here, this is a mmqcamera bug, ignore it Jan 01 00:02:26 apq8096 bash[3798]: ------ voxl-camera-server INFO: Stopping tracking camera Jan 01 00:02:26 apq8096 bash[3798]: ------ voxl-camera-server ERROR: Client connected to invalid channel: 0 Jan 01 00:02:26 apq8096 bash[3798]: ------ voxl-camera-server ERROR: Client connected to invalid channel: 0 Jan 01 00:02:26 apq8096 bash[3798]: ------ voxl-camera-server INFO: tracking camera stopped successfully Jan 01 00:02:26 apq8096 bash[3798]: ------ voxl-camera-server INFO: Stopping tof camera Jan 01 00:02:26 apq8096 bash[3798]: Fault address: 0x68 Jan 01 00:02:26 apq8096 bash[3798]: Address not mapped. Jan 01 00:02:26 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Main process exited, code=killed, status=11/SEGV[[0m Jan 01 00:02:26 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Unit entered failed state.[[0m Jan 01 00:02:26 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Failed with result 'signal'.[[0m Jan 01 00:03:04 apq8096 systemd[1]: Stopped voxl-camera-server. Jan 01 00:03:04 apq8096 systemd[1]: Started voxl-camera-server. Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server ERROR: Client connected to invalid channel: 3 Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server INFO: Camera server is now stopping Jan 01 00:03:04 apq8096 bash[4165]: There is a chance that it may segfault here, this is a mmqcamera bug, ignore it Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server INFO: Stopping tracking camera Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server ERROR: Client connected to invalid channel: 0 Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server ERROR: Client connected to invalid channel: 0 Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server INFO: tracking camera stopped successfully Jan 01 00:03:04 apq8096 bash[4165]: ------ voxl-camera-server INFO: Stopping tof camera Jan 01 00:03:04 apq8096 bash[4165]: Fault address: 0x68 Jan 01 00:03:04 apq8096 bash[4165]: Address not mapped. Jan 01 00:03:04 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Main process exited, code=killed, status=11/SEGV[[0m Jan 01 00:03:04 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Unit entered failed state.[[0m Jan 01 00:03:04 apq8096 systemd[1]: [[1;39mvoxl-camera-server.service: Failed with result 'signal'.[[0mThe line with the error about the
client connecting to an invalid channel: 0seems relevant. Could this be another SW version mismatch (similar to the forum post mentioned at the outset)?Any guidance on how to fix this?
-
Hey Jared,
Could you share the output of running:
voxl-camera-server -d 0Also, can you confirm that the sensors are plugged into the correct ports per our docs?
-
Here's the output you asked for:
yocto:/data$ voxl-camera-server -d 0 ------ voxl-camera-server INFO: Port J2 ------ voxl-camera-server: Done configuring hires camera Port : port_J2 Name : hires Enabled : 1 Type : hires Api : hal3 P-W : 640 P-H : 480 P-Fmt : nv21 FPS : 30 OverrideId : -1 AEAlgo : isp ------ voxl-camera-server INFO: Port J3 ------ voxl-camera-server: Done configuring tof camera Port : port_J3 Name : tof Enabled : 1 Type : tof Api : hal3 P-W : 224 P-H : 1557 P-Fmt : blob FPS : 15 Tof Mode : 9 OverrideId : -1 AEAlgo : isp ------ voxl-camera-server INFO: Port J4 ------ voxl-camera-server: Done configuring tracking camera Port : port_J4 Name : tracking Enabled : 1 Type : tracking Api : hal3 P-W : 640 P-H : 480 P-Fmt : raw8 FPS : 30 OverrideId : -1 AEAlgo : modalai =================MODALAI Auto Exposure Settings================== gain_min: 0 gain_max: 1000 exposure_min_us: 20 exposure_max_us: 33000 desired_msv: 58.000000 k_p_ns: 32000.000000 k_i_ns: 20.000000 max_i: 250.000000 p_good_thresh: 3 exposure_period: 1 gain_period: 2 display_debug: no exposure_offset_for_gain_calc: 8000 ================================================================= Writing new configuration to /etc/modalai/voxl-camera-server.conf Created pipe: hires channel: 0 Created pipe: tof channel: 6 Created pipe: tracking channel: 7 Starting Camera: tracking SUCCESS: Camera module opened ----------- Number of cameras: 2 ----------- HiRes camera id: 0 ----------- Mono camera id: 1 Available resolutions for camera: tracking: 640 x 480 320 x 240 Resolution: 640 x 480 Found! ------ voxl-camera-server: Preview buffer allocations Internal Exposure for: tracking 7 /builds/voxl-public/modal-pipe-architecture/voxl-camera-server/src/expgain_interface/expgain_interface_factory.cpp 45 Creating exposure interface Creating ModalAI exposure interface Initialize Initialize Port : port_J4 Name : tracking Enabled : 1 Type : tracking Api : hal3 P-W : 640 P-H : 480 P-Fmt : raw8 FPS : 30 OverrideId : -1 AEAlgo : modalai =================MODALAI Auto Exposure Settings================== gain_min: 0 gain_max: 1000 exposure_min_us: 20 exposure_max_us: 33000 desired_msv: 58.000000 k_p_ns: 32000.000000 k_i_ns: 20.000000 max_i: 250.000000 p_good_thresh: 3 exposure_period: 1 gain_period: 2 display_debug: no exposure_offset_for_gain_calc: 8000 ================================================================= Starting Camera: tof ------ voxl-camera-server INFO: Camera server is now stopping There is a chance that it may segfault here, this is a mmqcamera bug, ignore it ------ voxl-camera-server INFO: Stopping tracking camera ------ Result thread on camera: tracking recieved stop command, exiting ------ voxl-camera-server INFO: tracking camera stopped successfully ------ voxl-camera-server INFO: Stopping tof camera Fault address: 0x68 Address not mapped. Segmentation faultMy VOXL is buried deep in a drone. I'd have to disassemble the drone to view the ports. That said, the cables/connections haven't changed since last week when it was working. I presume they were connected properly then.
-
It seems that the TOF module is not detected:
SUCCESS: Camera module opened ----------- Number of cameras: 2 ----------- HiRes camera id: 0 ----------- Mono camera id: 1VOXL is only detecting two cameras, the hires and tracking, which is why those configurations are working correctly and the selected one fails. If the TOF module was setup properly, there would be a third camera listed, as in:
SUCCESS: Camera module opened ----------- Number of cameras: 3 ----------- ToF camera id: 0 ----------- Stereo camera id: 2 ----------- Mono camera id: 1The TOF module may have been disconnected or a cable could have been damaged since you last tested, as the camera should be detected even if voxl-camera-server is configured incorrectly.
-
I see.
Looks like I will need to take apart the drone to see if it somehow became disconnected. If that doesn't solve it, I'll try replacing it. I'll post an update (in a few days) once I have done both of those things.
Thanks again, @Matt-Turi.
-
@Matt-Turi, I check whether any connections were loose. There were none. I re-seated all the connectors anyway, but it made no difference.
I ended up purchasing a new TOF sensor. After I installed it, it worked! I am able to see the TOF outputs in
voxl-portal.I have one last issue, which could be related to the VOXL SW, so I'm keeping it in this thread instead of starting another one. After confirming the TOF sensor was working properly in
voxl-portal, I wanted to see the pointcloud in rviz.I followed the steps described here to view the TOF sensor data in rviz. On a companion linux PC, I would connect to the VOXL over wifi. I could successfully ping going both ways. I ran the appropriate export statements (as described in the TOF sensor User Guide). But when I tried to run
rvizorrostopic list, I would get the error messageERROR: Unable to communicate with master!. I tried this with two companion Linux PC's (Ubuntu 18.04 with ROS melodic and Ubuntu 20.04 with ROS noetic) and got the same results on both.It seems very much a networking issue. In searching ofther ROS forums, many attribute this error to a firewall being enabled. I've disabled the firewall on both companion Linux PC's but that didn't resolve the issue.
I'm scratching my head because it appears that everything is fine network-wise, but ROS is unable to connect. It seems very much like a ROS/networking issue (and I still think that is what it is), but I am at the point where I figure it wouldn't hurt to ask the following question: is there anything in the latest ModalAI SW that could be affecting this? If not, is there anything obvious I could be missing? Thanks in advance!
-
I figured out my issue.
On the VOXL, I had
ROS_HOSTNAME=localhost. When I ranunset ROS_HOSTNAME, the companion PC was able to connect properly. -
Yep,
This was an bug with voxl-utils v0.8.4 sourcing ros files in the wrong order, I just pushed a patch (0.8.5) up that fixes this, should be available in a few minutes.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login