Compatible ToF sensor setup with VOXL2 Mini
-
Yes the old TOF (V1) is EOL and we only have the TOF V2 available now, but due to increase power requirements, the connector had to change. The Starling Front End board (M0173) provides the connector to plug in the M0178 tof assembly (look at https://docs.modalai.com/voxl2-coax-camera-bundles/ C26, or C27).
The Starling camera front end is only compatible with VOXL2, not VOXL2 mini. However, if you just wanted to plug in TOF V2 directly into VOX2 Mini, we now have M0172, which you can find a pictured here : https://forum.modalai.com/topic/4125/indoor-navigation-using-4-depth-sensors . M0172 splits the VOXL2 dual camera connector into a TOF-compatible 40-pin molex connector and the uCoax connector for the cameras that use uCoax.
We don't have M0172 officially available, but if you want to try it we can make it available in the shop. So you would need (M0172 adapter + MSU-M0178-1-01 (the whole TOF sensor assembly) ).
The connection order would look like this:
VOXL2 (mini) <-> M0172 <-> M0177 <-> M0178 <-> actual TOF sensor
(everything to the right of M0172 is sold in this kit : https://www.modalai.com/products/m0178?variant=48528287793456)This would work for either VOXL2 or VOXL2 mini. The only down side i see is that M0177 is pretty short. I am not sure if we have an extension available, but will check.
EDIT: Here is the extension that can go between M0172 and M0177 : https://www.modalai.com/products/msu-m0170
Alex
-
@Alex-Kushleyev Thanks for getting back so quickly! We just have a couple of follow-up questions:
If we use this M0172 are we still able to have a tracking camera running? Such that the total setup looks something like: VOXL2 mini w/ M0178 ToF and M0014 tracking camera. If yes, then it would be great if the M0172 could be made available in the shop
-
@Alex-Kushleyev Hi again Alex, sorry for disturb just wanted to hear if you had seen the previous message? And if you could make the M0172 available?
-
Hi @Morten-Nissov ,
Sorry for the delay. We will get the M0172 board available for purchase ASAP.
Since your configuration uses M0014 camera, which would not be able to plug into M0172 splitter, here is your connection option:
- VOXL2 Mini J6:
- M0076 or M0135 interposer + M0014 camera
- VOXL2 Mini J7:
- M0172 interposer + TOF
(You could swap the connections, such as TOF is connected to J6, and vice versa). You would simply need to update the sensormodules (camera drivers) used, according to the slot id of each camera.)
When using VOXL2 mini with TOF, our guidance is to use only one TOF V2 per VOXL2 mini
- the TOF V2 sensor uses 5V power input, however VOXL 2 Mini only has a budget of 900mA on the 5V rail because the input voltage to the mini is 3.8V and there is a step-up regulator to 5.0V. This is different from VOXL2, which uses an APM / ESC to supply 5V 6A into VOXL2.
- the TOF V2 internally uses 3.3V and the pulsed current draw in different operating modes can be found here : https://docs.modalai.com/M0178/#irs2975-operating-modes . Using multiple TOF sensors on voxl2 mini can result in exceeding the current limit on the 5V regulator.
Keep in mind that you will still be able to plug in additional cameras if you need to:
- a uCoax camera plugged into M0172 (M0161, M0166, etc)
- any camera that uses older connector plugged into M0135 (M0014, M0107, etc)
One thing to keep in mind when using M0076 and M0135 is because they were originally designed for VOXL2, though compatible with VOXL2 mini, the interposers may interfere with the mini's mounting holes, because M0076 / M0135 (when inserted) have slight overlap with the mounting holes. However, the overlap may not be an issue if you dont have stand-offs on the side where M0076 / M0135 plugs into VOXL2 mini.
We will update this thread once M0172 is available in the shop.
Alex
- VOXL2 Mini J6:
-
@Alex-Kushleyev @Morten-Nissov , The team just posted the M0172 for sale.
Sorry for the delays....
https://www.modalai.com/products/mdk-m0172-1-00 -
@Vinny Sorry thought I had responded, thanks!
-
Hi,
We are trying to configure Voxl 2 Mini with the ToF sensor and a tracking camera with the , and we're having trouble with getting the ToF sensor to work. This is with the new ToF sensor and the new interposer you suggested for us to use.
The cameras are attached in the following way:
J7 <-> M0135 (JL) <-> M0014
J6 <-> M0172 (J1L) <-> M0170 <-> tof
The voxl suite we have on the board is: voxl-suite: 1.5.0To configure the cameras we run:
voxl-configure-sku
choosing options 23 (voxl-configure-sku) and 5 (pmd-tof ov7251 tracking).
The output was the following:Your answers would construct the following SKU: family code: MCCA-M0104 (voxl2-mini-board-only (MCCA-M0104)) compute board: 6 (voxl2-mini) hw version: 1 cam config: 05 modem config: () tx config: () extras config: () SKU: MCCA-M0104-C05-T0-M0-X0 This sku will be saved to persistent memory so VOXL will remember what it is installed in, even between system image flashes.So we run:
voxl-configure-mpaandrebootTo inspect the cameras, we run
voxl-inspect-cam -a, but we were able to see only the tracking camera:| Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fps | mbps | format | tracking | 307200 | 640 | 480 | 2.51 | 51 | 204 | 11.5 | 30.0 | 73.7 | RAW8So we restarted the service
voxl-camera-server, but we got the following error + warnings:existing instance of voxl-camera-server found, attempting to stop it ================================================================= configuration for 2 cameras: cam #0 name: tof sensor type: pmd-tof isEnabled: 1 camId: 0 camId2: -1 fps: 5 en_rotate: 1 en_rotate2: 0 en_preview: 1 pre_width: 224 pre_height: 1557 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: off msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 5 independent_exposure:0 cam #1 name: tracking sensor type: ov7251 isEnabled: 1 camId: 1 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 640 pre_height: 480 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: lme_msv msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 1 independent_exposure:0 fsync_en: 0 fsync_gpio: 109 ================================================================= thread is locked to cores: 4 5 6 7 connected to mavlink pipe Connected to cpu-monitor Starting Camera: tof (id #0) ERROR: Camera 0 failed to find supported preview config: 224x1557 WARNING: Failed to start cam tof due to invalid resolution WARNING: assuming cam is missing and trying to compensate Starting Camera: tracking (originally id #1) with id offset: 1 gbm_create_device(156): Info: backend name is: msm_drm setting ov7251 BLC register for cam id 0 ------ voxl-camera-server: Started 1 of 2 cameras ------ voxl-camera-server: Camera server is now runningNote, we also tried to modify the preview resolution, as well as disabling the preview entirely. But making these changes to the
/etc/modalai/voxl-camera-server.conffile didn't seem to change the actual configuration used, as reported when runningvoxl-camera-serverin terminal.Do you have any suggestions for how we can get this working?
-
Additional debugging:
Moreover, we test some other configurations:
- J7 <-> M0135 (JL) <-> M0014
- J6 <-> M0172 (J1L) <-> M0170 <-> tof
We copied the files:
com.qti.sensormodule.ov7251_*.binandcom.qti.sensormodule.irs2975c_*.bininto the folder:/usr/lib/camera
We used the command:camera-server-config-helper tracking:ov7251:0 tof:pmd-tof:2giving the following output:successfully wrote this camera config to disk: ================================================================= configuration for 2 cameras: cam #0 name: tracking sensor type: ov7251 isEnabled: 1 camId: 0 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 640 pre_height: 480 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: lme_msv msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 1 independent_exposure:0 cam #1 name: tof sensor type: pmd-tof isEnabled: 1 camId: 2 camId2: -1 fps: 5 en_rotate: 1 en_rotate2: 0 en_preview: 1 pre_width: 224 pre_height: 1557 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: off msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 5 independent_exposure:0 fsync_en: 0 fsync_gpio: 109 ================================================================= camera_server_config_helper is doneThen we run
voxl-camera-servergiving the following output:existing instance of voxl-camera-server found, attempting to stop it ================================================================= configuration for 2 cameras: cam #0 name: tracking sensor type: ov7251 isEnabled: 1 camId: 0 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 640 pre_height: 480 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: lme_msv msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 1 independent_exposure:0 cam #1 name: tof sensor type: pmd-tof isEnabled: 1 camId: 2 camId2: -1 fps: 5 en_rotate: 1 en_rotate2: 0 en_preview: 1 pre_width: 224 pre_height: 1557 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: off msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 5 independent_exposure:0 fsync_en: 0 fsync_gpio: 109 ================================================================= thread is locked to cores: 4 5 6 7 connected to mavlink pipe Connected to cpu-monitor Starting Camera: tracking (id #0) gbm_create_device(156): Info: backend name is: msm_drm setting ov7251 BLC register for cam id 0 Starting Camera: tof (id #2) WARNING: cam tof (id 2) does not seem to be alive ------ voxl-camera-server: Started 1 of 2 cameras ------ voxl-camera-server: Camera server is now runningAnd running
voxl-inspect-cam -astill gives the following output:| Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fps | mbps | format | tracking | 307200 | 640 | 480 | 2.28 | 51 | 294 | 11.5 | 30.0 | 73.7 | RAW8To be sure that all the parts were working, we tested two M0172 and 2 ToF sensors, which were actually working on VOXL 2. We also swapped the tracking camera and the ToF sensor (J6 <-> J7), copying the
.binfiles according to and adjusting the ID numbers. -
@Morten-Nissov , please only put one sensormodule with the correct slot number in
/usr/lib/camera. This will help eliminate any confusion.Also, does
voxl-camera-server -lactually detect your TOF module?It seems your camera id is actually incorrect. It should be 1. The slot id is 2 (which is tied to the hardware), but the software camera Id is assigned as the cameras are probed in each slot.
voxl-camera-server -lwill tell you (at the top) which cameras have been detected, which slot and camera ids.Alex
-
@Alex-Kushleyev Thanks for getting back to us so quickly.
We removed all the
com.qti.sensormodule.ov7251_*.binand thecom.qti.sensormodule.irs2975c_*.binfiles.
Then we ran, as you suggested, the command:voxl-camera-server -l. The output was:existing instance of voxl-camera-server found, attempting to stop it DEBUG: Attempting to open the hal module DEBUG: SUCCESS: Camera module opened on attempt 0 DEBUG: ----------- Number of cameras: 2 DEBUG: Cam idx: 0, Cam slot: 0, Slave Address: 0x00E2, Sensor Id: 0x7750 DEBUG: Cam idx: 1, Cam slot: 2, Slave Address: 0x007A, Sensor Id: 0x2975 DEBUG: Note: This list comes from the HAL module and may not be indicative DEBUG: of configurations that have full pipelines DEBUG: Number of cameras: 2 ==================================== Stats for camera: 0 ANDROID_SCALER_AVAILABLE_RAW_SIZES: These are likely supported by the sensor 640 x 480 640 x 480 640 x 480 ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS: These are NOT necessarily supported by the sensor 640 x 480 HAL_PIXEL_FORMAT_YCbCr_420_888 640 x 480 HAL_PIXEL_FORMAT_BLOB 640 x 400 HAL_PIXEL_FORMAT_YCbCr_420_888 640 x 400 HAL_PIXEL_FORMAT_BLOB 640 x 360 HAL_PIXEL_FORMAT_YCbCr_420_888 640 x 360 HAL_PIXEL_FORMAT_BLOB 352 x 288 HAL_PIXEL_FORMAT_YCbCr_420_888 352 x 288 HAL_PIXEL_FORMAT_BLOB 320 x 240 HAL_PIXEL_FORMAT_YCbCr_420_888 320 x 240 HAL_PIXEL_FORMAT_BLOB 240 x 320 HAL_PIXEL_FORMAT_YCbCr_420_888 240 x 320 HAL_PIXEL_FORMAT_BLOB 176 x 144 HAL_PIXEL_FORMAT_YCbCr_420_888 176 x 144 HAL_PIXEL_FORMAT_BLOB 640 x 480 HAL_PIXEL_FORMAT_RAW10 640 x 480 HAL_PIXEL_FORMAT_RAW12 640 x 480 HAL_PIXEL_FORMAT_RAW16 640 x 480 HAL_PIXEL_FORMAT_RAW_OPAQUE ANDROID_SENSOR_INFO_SENSITIVITY_RANGE min = 54 max = 3451 ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY 3451 ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE min = 0ns max = 1266732525ns ==================================== Stats for camera: 1 ANDROID_SCALER_AVAILABLE_RAW_SIZES: These are likely supported by the sensor 240 x 1629 ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS: These are NOT necessarily supported by the sensor 240 x 320 HAL_PIXEL_FORMAT_YCbCr_420_888 240 x 320 HAL_PIXEL_FORMAT_BLOB 176 x 144 HAL_PIXEL_FORMAT_YCbCr_420_888 176 x 144 HAL_PIXEL_FORMAT_BLOB 240 x 1629 HAL_PIXEL_FORMAT_RAW10 240 x 1629 HAL_PIXEL_FORMAT_RAW12 240 x 1629 HAL_PIXEL_FORMAT_RAW16 240 x 1629 HAL_PIXEL_FORMAT_RAW_OPAQUE ANDROID_SENSOR_INFO_SENSITIVITY_RANGE min = 54 max = 0 ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY 0 ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE min = 0ns max = 0ns ==================================== Number of cameras detected: 2 ====================================Given the output, we copied the files
com.qti.sensormodule.ov7251_0.binandcom.qti.sensormodule.irs2975c_2.bininto/usr/lib/camerafolder.
Then we ran:camera-server-config-helper tracking:ov7251:0 tof:pmd-tof:2and got the following output:successfully wrote this camera config to disk: ================================================================= configuration for 2 cameras: cam #0 name: tracking sensor type: ov7251 isEnabled: 1 camId: 0 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 640 pre_height: 480 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: lme_msv msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 1 independent_exposure:0 cam #1 name: tof sensor type: pmd-tof isEnabled: 1 camId: 2 camId2: -1 fps: 5 en_rotate: 1 en_rotate2: 0 en_preview: 1 pre_width: 224 pre_height: 1557 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: off msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 5 independent_exposure:0 fsync_en: 0 fsync_gpio: 109 ================================================================= camera_server_config_helper is doneFinally, we ran:
voxl-camera-server, and we got:existing instance of voxl-camera-server found, attempting to stop it ================================================================= configuration for 2 cameras: cam #0 name: tracking sensor type: ov7251 isEnabled: 1 camId: 0 camId2: -1 fps: 30 en_rotate: 0 en_rotate2: 0 en_preview: 1 pre_width: 640 pre_height: 480 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: lme_msv msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 1 independent_exposure:0 cam #1 name: tof sensor type: pmd-tof isEnabled: 1 camId: 2 camId2: -1 fps: 5 en_rotate: 1 en_rotate2: 0 en_preview: 1 pre_width: 224 pre_height: 1557 en_raw_preview: 1 en_small_video: 0 small_video_width: -1 small_video_height: -1 en_large_video: 0 large_video_width: -1 large_video_height: -1 en_misp: 0 misp_width: -1 misp_height: -1 en_snapshot: 0 snap_width: -1 snap_height: -1 exif_focal_length: 0.000000 exif_focal_len_35mm_format:0 exif_fnumber: 0.000000 ae_mode: off msv_exposure_min_us: 20 msv_exposure_max_us: 33000 gain_min : 54 gain_max : 8000 standby_enabled: 0 decimator: 5 independent_exposure:0 fsync_en: 0 fsync_gpio: 109 ================================================================= thread is locked to cores: 4 5 6 7 connected to mavlink pipe Connected to cpu-monitor Starting Camera: tracking (id #0) gbm_create_device(156): Info: backend name is: msm_drm setting ov7251 BLC register for cam id 0 Starting Camera: tof (id #2) WARNING: cam tof (id 2) does not seem to be alive ------ voxl-camera-server: Started 1 of 2 cameras ------ voxl-camera-server: Camera server is now runningJust to make sure that only one camera was working, we ran:
voxl-inspect-cam -aand got:| Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fps | mbt | tracking | 307200 | 640 | 480 | 5.04 | 111 | 179 | 11.6 | 30.0 | 73.7 | RAW8What do you think?
-
@Morten-Nissov said in Compatible ToF sensor setup with VOXL2 Mini:
DEBUG: Cam idx: 0, Cam slot: 0, Slave Address: 0x00E2, Sensor Id: 0x7750
DEBUG: Cam idx: 1, Cam slot: 2, Slave Address: 0x007A, Sensor Id: 0x2975Please use camera id 1 in your
voxl-camera-server.confinstead of 2. The slot # is 2 but camera id is 1.Alex
