@tkddnjs825 ,
Can you please run “voxl-camera-server -l” and look at the very top to see which cameras have been detected (this is regardless of what you enable in camera server config). We first want to make sure all the cameras are correctly detected (should be 4 cameras). You could past the first part that lists detected cameras, their ids and slot numbers.

@brianaustin , you probably do not have the correct sensormodule for your tracking camera in /usr/lib/camera/.

I can help you. Please let me know which camera ports all the cameras are plugged in (ov7251 stereo is probably in J6L, etc).

Also please provide the output of the following command:

ls /usr/lib/camera/*sensormodule*

You can see many examples of camera configurations and where the cameras are plugged into different connectors : https://docs.modalai.com/voxl2-camera-configs/

Finally to see which cameras are detected, you can run:
voxl-camera-server -l
this will detect the cameras and exit. at the very top you should see something like this, which lists detected cameras and their camera slot IDs. Sensor Id can tell us which camera type it is. In your case, since you are connecting 4 cameras (two in stereo, one tracking, one hires), you should see 4 cameras being detected if all works well. However, i think right now you will see 3. And the solution will be to copy the correct sensormodule file for ov7251 for the right slot number from /usr/share/modalai/chi-cdk/ov7251/ to /usr/lib/camera . If this is a standard camera config, you can just run voxl-configure-cameras.

This is my example of listing cameras:

voxl2-mini:/$ voxl-camera-server -l
DEBUG:   Attempting to open the hal module
DEBUG:   SUCCESS: Camera module opened on attempt 0
DEBUG:   ----------- Number of cameras: 3

DEBUG:   Cam idx: 0, Cam slot: 0, Slave Address: 0x0020, Sensor Id: 0x0214
DEBUG:   Cam idx: 1, Cam slot: 1, Slave Address: 0x0030, Sensor Id: 0x0356
DEBUG:   Cam idx: 2, Cam slot: 2, Slave Address: 0x0034, Sensor Id: 0x0577

Alex

Hi @m1baldwin , are you using VOXL1 and VOXL2 ?

https://docs.modalai.com/voxl2-mini-connectors/#j7---camera-group-1-specific-pinout has been updated to i2c-0 and SDA / SCL pins are labeled

BTW, you should be able to use M0076 single camera interposer or M0135 dual image interposer to access the I2C-0 pins if you need to:

https://docs.modalai.com/m0076/#2d-diagrams

• TP 5 and 6 are GPIO 4 and 5

https://docs.modalai.com/m0135/

• TP1 and TP2 are GPIO 4 and 5
  
  

@jcai , good news. I checked the kernel Device Tree for VOXL2 Mini (M0104) and I believe that J7 has /dev/i2c-0 , which is using GPIO 4 and 5. The /dev/i2c-0 is present on on the file system.

I have not tested the i2c port just now, but you should give it a go! Just watch the digital IO voltage levels, they are 1.8V coming off from the voxl2 / mini camera connectors. Not buffered, but ESD protected.

I think we need to update the docs to reflect that this is /dev/i2c-0 and not /dev/i2c-4.

https://gitlab.com/voxl-public/system-image-build/meta-voxl2-bsp/-/blob/qrb5165-ubun1.0-14.1a/recipes-kernel/linux-msm/files/dts/m0104/m0104-modalai-qupv3.dtsi#L11

https://gitlab.com/voxl-public/system-image-build/meta-voxl2-bsp/-/blob/qrb5165-ubun1.0-14.1a/recipes-kernel/linux-msm/files/dts/common/m0xxx-kona-pinctrl.dtsi?ref_type=heads#L3117

Alex

@griffin , please see the following post with the same symptom and fix : https://forum.modalai.com/topic/4171/frame-size-mismatch-got-43200-bytes-from-pipe-expected-38528

We will test this and merge to dev soon, meanwhile if you can re-build the camera server, you could test it.

Alex

@tkddnjs825 depending on which cameras are connected to your M0173, you can select the closest configuration using voxl-configure-cameras and select option 26, 27, or 28. Then you can manually edit /etc/modalai/voxl-camera-server.conf to disable the camera(s) that you don't have installed (you can just remove those camera config entries completely from the file to avoid confusion)

https://docs.modalai.com/voxl2-coax-camera-bundles/

Please let me know if you need any further clarification.

Alex

@jcai , Here is a draft of instructions for using the Hadron adapter (M0159 + M0181) for beta testing. I will put these up on our docs page but just wanted to provide these to you so you can test the hardware. Please let me know if you have any questions.

WARNING:

• even though the coax cables are the same for both connections, the order cannot be swapped
• HR connector has to conect to HR and 5L to 5L
• M0159 J2 -> M0181 J1
• M0159 J3 -> M0181 J2
• shipped assemblies have been correctly connected and tested

Software Setup

• use voxl2 mini (VOXL2 also supported)
• install sdk 1.4.0 (tested) or later
• change apt source to dev : /etc/apt/sources.list.d/modalai.list
  • deb [trusted=yes] http://voxl-packages.modalai.com/ ./dists/qrb5165/dev/binary-arm64/

apt update
apt install libmodal-pipe voxl-portal voxl-camera-server

Confirm sensormodule drivers exist

• in chi-cdk directory where all the camera drivers are kept (but not used)

ls /usr/share/modalai/chi-cdk/ov64b40 : com.qti.sensormodule.ov64b40_0.bin ... 5.bin
ls /usr/share/modalai/chi-cdk/boson:    com.qti.sensormodule.boson_0.bin ... 5.bin

Confirm these exist (also part of camera sernsor drivers):

/usr/lib/camera/com.qti.sensor.ov64b40.so
/usr/lib/camera/com.qti.sensor.boson.so

Connect M0181 to Voxl2 Mini J7

• make sure pin alignment on connectors is correct. the mounting holes should align and the tab of M0181 will hover above the main SoC
• see attached image
• the tab can be removed from M0181 if not needed

Copy correct sensormodule drivers

cp /usr/share/modalai/chi-cdk/boson/com.qti.sensormodule.boson_2.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/ov64b40/com.qti.sensormodule.ov64b40_3.bin /usr/lib/camera/

Configure CCI Mux on M0159 via GPIO

• Use gpio 6 on J7 of VOXL2 Mini to configure the i2c mux to connect J7U cci to Hadron's cam cci
• this needs to be done each time after reboot, before running camera server

voxl-gpio -m 6 out && voxl-gpio -w 6 1

Detect cameras

voxl-camera-server -l
(should detect both cameras)

Supported resolutions / modes

• OV64B currently supports the following modes
• these are the RAW modes that camera can be configured to (by the camera pipeline)
  • the camera pipeline picks the best mode based on the requested resolution and fps
  • note that if you request 3840x2160 30FPS, the camera pipeline actually selects 4624x3472 because it matches the desired FPS (30)
  • actual selected mode can be checked using logcat | grep -i selected before starting camera server, the selected resolution will be printed for each camera.

# (Mbps per csi lane)
Mode0_9248x6944_10fps_2500.8Mbps
Mode1_4624x3472_30fps_PD_1136x860_1502Mbps
Mode4_3840x2160_60fps_2500.8Mbps
Mode5_1920x1080_240fps_2500.8Mbps
Mode6_1920x1080_30fps_800Mbps

Max resolution (works! about 9fps)

		"preview_width":	9248,
		"preview_height":	6944,

• Note that at this >8K resolution, the Qualcomm ISP runs in dual VFE mode, which means you cannot use any other non-RAW camera that uses the ISP.
• MISP approach does not use Qualcomm ISP and will support more cameras (documentation coming soon)

Minimum config

• paste into /etc/modalai/voxl-camera-server.conf:

{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[
		{
			"type":	"boson",
			"name":	"boson",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	true,
			"en_misp":	false,
			"preview_width":	640,
			"preview_height":	512,
			"en_raw_preview":	true,
			"en_small_video":	false,
			"en_large_video":	false,
			"ae_mode":	"off"
		},
		{
			"type":				"ov64b",
			"name":				"hires",
			"enabled":			true,
			"camera_id":		1,
			"fps":				30,
			"en_preview":		true,
			"en_misp":			false,
			"preview_width":	1920,
			"preview_height":	1080,
			"en_raw_preview":	false,
			"en_small_video":	false,
			"en_large_video":	false,
			"en_snapshot":	false,
			"ae_mode":	"isp",
			"gain_min":	54,
			"gain_max":	32000
		}
		]
}

Start Camera Server

• run voxl-camera-server in foreground to make sure everything is going right and you can view streams using voxl-portal
• note that very high resolution streams viewed as YUVs (which are transferred as MJPG by voxl-portal) will take a lot of CPU to encode to MJPG and also a lot of bandwidth to stream.

Image of M0181 Plugged into VOXL2 Mini J7

@jcai , can you please let me know where you see i2c0 on J7 of voxl2 mini? Please keep in mind that CCI_I2C ports are reserved for camera use.

Regarding accessing QUP0 on J19 from application processor, it not currently supported. It would require a bridge application to communicate the the DSP's I2C port. It's on our todo list to look at enabling this, but there is no timeline at this moment. The best bet right now would be writing a custom PX4 driver that will run on the DSP and talk to the device and then somehow pass that information to the application processor. There are a number of examples of i2c device drivers, such as barometer, compass drivers that are running on DSP.

Alex

@smilon , Can you please clarify what this is a picture of? I don't recognize it from this angle.

Alex

@jameskuesel ,

Can you please provide a bit more detail about your test -- when you enable M0161, how exactly do you test it? Do you view the images using voxl-portal? and if so, how is your starling2 connected to the network? If you are using the wifi dongle, then we have noticed that moving the wifi dongle away from the GPS antenna helps reduce interference. You can test if the interference is still present by unplugging the wifi dongle completely from usb port and use voxl-inspect-cam test application to make sure the camera is running (inspect the same camera stream that you were previously viewing).

Also, please let me know what resolution / FPS you are testing the camera at.

Regarding the MIPI clock rates, i would need to double check, but i believe the IMX412 driver uses 1.5Gbps MIPI bit rate, which is close to GPS L1, but not really close that it would cause interference. We can definitely try different bit rates to see if it helps reduce EMI (if that is indeed the case), but let's first rule out the potential wifi dongle issue. We have ability to change the MIPI clock rate in quite a wide range, but this will require a different sensormodule driver for the camera, which I would need to provide to you.

Another thing, we recently found that the Starling Front End M0173 board may cause GPS interference via the Lepton connection, so if you have that plugged in, please unplug and see if that helps with GPS signal. Lepton needs 25Mhz clock, so one of the harmonics could hit exactly GPS L1 at 1575Mhz.

Alex

@ashwin , from software point of view, running multiple TOF V2 sensors on VOXL2 is not an issue, but as Vinny said, the power requirements would put a limit of a single TOF sensor per VOXL camera connector (J6, J7, J8), so that would mean at most 3 TOF sensors connected to J6L, J7L and J8L (camera slots 0, 2, 4 in order to avoid i2c slave address conflicts -- this is more of a note to self ).

However, assuming the SW can support these 3 TOF sensors, i am not sure we actually have the required interposers / flex cables to connect 3 TOF V2 sensors to the 3 camera ports. @Vinny , what do you think?

Also, @ashwin , would you need any other cameras connected to VOXL2 besides the multiple TOF sensors?

Alex

and also a simple example here : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/master/tools/voxl-inspect-tof.c?ref_type=heads

@jameskuesel , thank you for letting me know about the focus issue. It seems we had a few cameras shipped out without being properly focused - we will make sure to take care of it in the future. Sorry about that.

Regarding the image quality, after adjusting the lens focus, are you satisfied with the image resolution / quality? Also i believe the color balance outdoors should look better under natural light.

Alex

I just made a fix on a branch, but did not test it:

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commit/43aa51a43c339e008b86d637b87cb22d9c369588

If you are able to test the fix, please give it a try and let me know if it works!

Thanks!

Alex

@eric , thanks again for your input on this, i have posted a complete tutorial how to enable OpenCL in Docker on VOXL2 : https://docs.modalai.com/voxl-2-opencl-in-docker/

Alex

Hi @LucaVertiq , this is cool stuff. I am curious if you were able to do any direct comparisons of qvio performance using unfiltered vs low pass (rpm-based) vs notch (rpm based)?

In order to replicate the same conditions for each test, this would have to be done using mpa playback after collecting mpa logs of just imu data and images.

since the rpm data is not coming from mpa, it cannot be logged / played back using our logger.

However, one idea is to process the IMU data logs using RPM logs using a separate script and then run the playback + qvio to generate new results. The IMU data processing can have different options like the rpm-based notch filter and it would obviously also need to have a px4 log that contains the ESC status.

This also leads to a question, whether the IMU data should be filtered in the imu server or the application that uses the imu, since the requirements for the IMU data may differ for different applications, so it may be best to leave the IMU server publishing the original data. But also, there could be a separate output pipe from the imu sever with the notch filtered imu data..

Anyway, just wanted to check if you happened to get any quantitative results from your testing that show how the filtering helps with QVIO performance. My understanding from working with QVIO in the past was that the best thing to do was to feed unfiltered imu data into QVIO and let it integrate the noisy data. low pass filtering the IMU data too much would actually remove some of the actual components of motion that are important for motion tracking. So i am curious what exactly has improved in your case when you added notch filtering - is the drift vs distance improved or the number of VIO "blow-ups" has reduced?

Lots of questions, but I am happy to discuss this further and see how we can improve performance of our VIO stack

Alex

@Jetson-Nano , As far as I know, turtle mode should work the same when running PX4 on VOXL2 and FC V2. Have you tested it? The LED command should not interfere with the turtle mode.

I have not actually tested the turtle mode on FC myself, but the code that is running in voxl-esc px4 driver is the same, so as long as you are entering the turtle mode correctly, it should work.

Alex

@Jetson-Nano , there is probably more than one way of getting this to work, but I have been told that with the new approach that we have just implemented (being tested now), the mavlink tunnel should take care of routing the LED data from VOXL2 to FC V2 to ESC.

I asked my colleague to help describing how to test this.

Alex