ADB will not find Flight Deck if the tracking camera is connected (J4)

DillonAllen

I'm currently trying to test whether the cameras on our Flight deck are working or not but I could not get the adb to detect any devices until I decided to start unplugging cameras until it would allow me to connect. What I found out is that the Tracking Camera (J4) is causing this problem and that anytime this camera is plugged in adb will not detect the Flight Deck.

tom

@DillonAllen If any of the cameras are plugged in to the wrong port or are plugged in backwards then the VOXL will fail to boot. Since the tracking camera does belong on J4 it is most likely being plugged in backward.

DillonAllen

@tom Wow! it was that simple. I guess I thought I had it in right since in this configuration the cable has to flow backwards. As a follow up question, when I run voxl-rtsp I get
~$ adb shell voxl-rtsp -c 0
voxl-rtsp version 1.0.3
Setting camera number to: 0
Connected
Unable to start camera: -22
Disconnected
voxl-rtsp exiting

And I get that for each camera (0-2)

Also thank you for your quick response time!

tom

@DillonAllen Are you using the latest software release? You can check this with voxl-version.

Unless you have a specific reason for using voxl-rtsp, I would recommend using a combo of voxl-camera-server and voxl-streamer as they are newer and much more dynamic than voxl-rtsp.

You can find documentation for voxl-camera-server here: https://gitlab.com/voxl-public/modal-pipe-architecture/voxl-camera-server

And docs for voxl-streamer here: https://docs.modalai.com/voxl-streamer/

DillonAllen

@tom So I have Voxl-camera-server 0.5.7 installed by looking at opkg list-installed, and I'm fairly confident I have the updated emulator installed as well and I am able to run the voxl-streamer -c video-test and I am able to connect via softap, but the problem is that when I try to connect to any of the other cameras I get this same error:
yocto:/# voxl-streamer -c stereo-mpa
Using configuration stereo-mpa
Using configuration file /etc/modalai/voxl-streamer.conf
ERROR in pipe_client_init_channel opening request pipe: No such device or address
Most likely the server stopped without cleaning up
Client is cleaning up pipes for the server

And it will just sit at cleaning pipes until I close the terminal. Appreciate all the help so far!

When you connect or disconnect a camera (making sure to only do so when the board is not powered), you need to re run the voxl-configure-cameras script and make sure that you provide the correct camera configuration. Also, make sure that the cameras are plugged into the correct ports, as they will not all work in all of the ports.

Additionally, there have been a number of stability and performance improvements since camera server 0.5.7, I'd strongly recommend reflashing your platform with the platform release 330-046 found here. This will wipe the drone except for optionally the data partition, so make sure that you back up any data not there.

DillonAllen

@Alex-Gardner I'd rather not reflash my platform since I'm more worried that I will do more harm than good and also because I believed that the flight deck came nearly ready with only needing to do some firmware updates and configuration but if you think that reflashing would be beneficial and help fix some problems I could. I think I'm misunderstanding something with the voxl-streamer, I'm running the voxl-configure-cameras 3 script to get the default configuration (I'm sure all of the cameras are plugged in to the correct ports, except I do have an MIPI extension running to J2 and J3) and I'm getting told that they are writing the configuration to etc/modalai/voxl-camera-server.conf but when I run voxl-streamer -c tracking
I get told that it is using configuration file /etc/modalai/voxl-streamer.conf and I get the error
"ERROR: object missing tracking"
And if I try to use the file that its written to I get:
yocto:/# voxl-streamer -f /etc/modalai/voxl-camera-server.conf
Using configuration file /etc/modalai/voxl-camera-server.conf
Using configuration file /etc/modalai/voxl-camera-server.conf
ERROR: object missing configuration
Failed to get configuration name from configuration file
ERROR: Could not parse the configuration data

Yes, the flight deck came ready to use with our full software bundle, but we've since released a new system image and voxl-suite with new tools and patches that, while not required to have a functional system, we encourage people to use since there are bugfixes, improvements, and this is the version that we all use in our lab so it'll be easier for us to help debug if something is awry. I'm pretty sure that your issue here is that the tracking voxl-streamer configuration is the name in our newest software release, and if you're running voxl-suite 0.3.4 the configuration name is (I believe) tracking-mpa. You can check for certain by looking at

/etc/modalai/voxl-streamer.conf

voxl-configure-cameras sets up the voxl-camera-server config file (which is different from streamer). Camera server is our driver that pulls frames from the device and publishes them through the file system through FIFO pipes (you can see what data is being published at /run/mpa). Streamer takes the frames from these pipes and publishes them through GStreamer.

DillonAllen

Okay, so now I have reflashed my system with the most updated version and have reconfigured everthing that I believe that I needed to, but the problem now is that when I run the voxl-streamer command I get stuck at "Using configuration file /etc/modalai/voxl-streamer.conf" like this:
yocto:/$ voxl-streamer
Using configuration file /etc/modalai/voxl-streamer.conf

and it does the same for any of the other cameras that I try.

So now you can run it with any of the configuration options specified in that file such as:

voxl-streamer -c tracking

for tracking or

voxl-streamer -c stereo

for the stereo cameras

DillonAllen

Yes, but when I try to run it I just get stuck at "Using configuration file ..."

yocto:/$ voxl-streamer -c tracking
Using configuration tracking
Using configuration file /etc/modalai/voxl-streamer.conf

And I've left it for almost 5 minutes with it stuck at this stage.

Is voxl-camera-server running? You can check with

voxl-inspect-services

DillonAllen

It is enabled but not running, so how would I go about running it?

yocto:/$ voxl-inspect-services
 Service Name        |  Enabled  |   Running   |  CPU Usage
--------------------------------------------------------------
 docker-autorun      | Disabled  | Not Running | Not Running
 docker-daemon       | Disabled  | Not Running | Not Running
 modallink-relink    | Disabled  | Not Running | Not Running
 voxl-camera-server  |  Enabled  | Not Running | Not Running

You can run it with:

systemctl start voxl-camera-server

If it is enabled but not running though that means that it crashed on startup (or you stopped it). If you didn't manually stop it that means it encountered an error which is usually due to a bad camera connector causing the driver to throw an error. If you start it and it still shows up as stopped afterwards could you run it manually with:

voxl-camera-server -d 0

and paste the output here

DillonAllen

Here is the output:

yocto:/$ voxl-camera-server -d 0
------ voxl-camera-server INFO: Port J2

------ voxl-camera-server: Done configuring hires camera
	 Port       : port_J2
	 Name       : hires
	 Enabled    : 1
	 Type       : hires
	 Api        : hal3
	 P-W        : 640
	 P-H        : 480
	 P-Fmt      : nv21
	 FPS        : 30
	 OverrideId : -1
	 AEAlgo     : isp

------ voxl-camera-server INFO: Port J3

------ voxl-camera-server: Done configuring stereo camera
	 Port       : port_J3
	 Name       : stereo
	 Enabled    : 1
	 Type       : stereo
	 Api        : hal3
	 P-W        : 1280
	 P-H        : 480
	 P-Fmt      : nv21
	 FPS        : 30
	 OverrideId : -1
	 AEAlgo     : modalai
=================MODALAI Auto Exposure Settings==================
gain_min:                        0
gain_max:                        1000
exposure_min_us:                 100
exposure_max_us:                 33000
desired_msv:                     58.000000
k_p_ns:                          32000.000000
k_i_ns:                          20.000000
max_i:                           250.000000
p_good_thresh:                   3
exposure_period:                 1
gain_period:                     2
display_debug:                   no
exposure_offset_for_gain_calc:   8000
=================================================================

------ voxl-camera-server INFO: Port J4

------ voxl-camera-server: Done configuring tracking camera
	 Port       : port_J4
	 Name       : tracking
	 Enabled    : 1
	 Type       : tracking
	 Api        : hal3
	 P-W        : 640
	 P-H        : 480
	 P-Fmt      : raw8
	 FPS        : 30
	 OverrideId : -1
	 AEAlgo     : modalai
=================MODALAI Auto Exposure Settings==================
gain_min:                        0
gain_max:                        1000
exposure_min_us:                 100
exposure_max_us:                 33000
desired_msv:                     58.000000
k_p_ns:                          32000.000000
k_i_ns:                          20.000000
max_i:                           250.000000
p_good_thresh:                   3
exposure_period:                 1
gain_period:                     2
display_debug:                   no
exposure_offset_for_gain_calc:   8000
=================================================================

Writing new configuration to /etc/modalai/voxl-camera-server.conf
Created pipe: hires_preview channel: 0
Created pipe: stereo channel: 1
Created pipe: tracking channel: 2
Starting Camera: tracking
SUCCESS: Camera module opened
----------- Number of cameras: 0


------ voxl-camera-server INFO: Camera server is now stopping
		There is a chance that it may segfault here, this is a mmqcamera bug, ignore it

------ voxl-camera-server INFO: Stopping tracking camera
Fault address: 0x68
Address not mapped.
Segmentation fault

I do have the J3 and J2 ports using the MIPI extension cables if that might be causing problems

You have all of the cameras indicated there plugged in (hires, stereo, tracking)?

DillonAllen

Yes, and I've double checked that they are all in the correct ports

It's also possible that your extension cables are plugged in backwards, 90% of that output is reading the config file at /etc/modalai/voxl-camera-server.conf. The critical part is the

----------- Number of cameras: 0

This is the response from the driver when we asked how many cameras it sees, it will often refuse to open any of them if any of the lines are incorrect, could you show a picture of you cable setup?

There could also be a break in one of the cable lines (they can bend gently but should not be folded to an angle)

DillonAllen

Sorry for the late response, I've been doing some testing. so I've narrowed it down to the error lying in the stereo cameras, I connected each sensor one at a time, reconfigured, and ran the voxl-streamer and was able to get footage for the hires and tracking, but when it came to the stereo it would not work. When I would reboot the system and inspect the services I would see that the voxl-camera-server is running, but once I tried to voxl-streamer -c stereo it would get stuck at the same place and the voxl-camera-server would stop running. I've confirmed that both of my extension cables are working by testing them with the hires camera. I checked the physcial connections on the stereo cameras and there are no creases in them, they look just as they were folded with the original setup, and I believe that if any of the connections were inverted then the VOXL wouldn't even start up.