@dlee ,

Yes the new TOF sensor (IRS2975C) is more powerful that the previous generation. What I mean by that is that it can emit more IR power but also heats up more. Emitting more power allows the sensor detect objects at larger distances or objects that are not as reflective.

In current operating mode, the auto exposure control is enabled inside the sensor itself, which modulates the emitted IR power based on the returns that the sensor is getting. That is to say, the power draw will vary depending on what is in the view of the sensor. If there are obstacles nearby, the output power should be low, otherwise it can be high. At full power, the module can consume close to 0.8-0.9W

So the first solution, if design allows, is to add a heat spreader to dissipate the heat, which you already started experimenting with. The sensor has a large exposed copper pad in the back for heat sinking purposes for this exact reason. Just be careful not to short this pad to anything, use non-conducting (but heat transfering) adhesive pad between the sensor and heat spreader.

In terms of a software solution to the issue, we can query the temperature of the emitter. We can also control the maximum emitted power used by the auto exposure algorithm. That is to say, still leave the auto exposure running in the sensor, but limit the maximum power that it is allowed to use.

We are planning to add some software protection that limits the maximum output power as a function of the emitter temperature. This will require some implementation and testing.

Meanwhile, please consider using a heat spreader, which will be the best solution if you want to make use of the full sensor's operating range and not have our software limit the output power in order to prevent overheating.

Hello @Kashish-Garg-0

we have a curve that is "motor voltage vs rpm", meaning that for a desired RPM, it tells the ESC what average motor voltage should be applied. The average motor voltage is defined as battery_voltage * motor_pmw_duty_cycle. The battery voltage in this curve is in millivolts. Since you are typically controlling the desired RPM, as a user you do not need to worry about what "throttle" or voltage to apply - the ESC does this automatically in order to achieve the desired RPM. this calibration curve is used as a feed-forward term in the RPM controller. The ESC does support an "open loop" type of control where you specify the power from 0 to 100%, which is similar to a standard ESC, but PX4 does not use that ESC control mode.

By the way, you can test the ESC directly (not using PX4) using our voxl-esc tools (https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/master/voxl-esc-tools) which works directly on VOXL2 or a standalone linux PC (or mac). voxl-esc-spin.py has a --power argument where you specify the power from 0 to 100, which translates directly to the average duty cycle applied to the motor.

Here is the calibration for the Starling V2 motor / propeller that we use:
https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/mavic_mini_2/mavic_mini_2.xml?ref_type=heads#L63

Also, you can take a look at this post to see how to interpret those parameters a0, a1, a2 : https://forum.modalai.com/topic/2522/esc-calibration/2

We also have some dyno tests for this motor / propeller : https://gitlab.com/voxl-public/flight-core-px4/dyno_data/-/blob/master/data/mavic_mini2_timing_test/mavic_mini2_modal_esc_pusher_7.4V_timing0.csv . We are not sure how accurate that is, but it can be used as a starting point. @James-Strawson can you please confirm that is the correct dyno data for the Starling V2 motors?

Alex

@reber34 , perhaps this approach can work for you:

• record a video encoded at high bit rate (using voxl-camera-server and voxl-record-video . Please note that the output of voxl-record-video will not be in a standard container (such as mp4, etc), but you can fix it with ffpeg : ffmpeg -r 30 -i voxl-record-video.h264 -codec copy videofile.mp4
• re-encode the video offline with desired codecs / bit rates / resolutions
• install gst-rtsp-launch which uses gstreamer to set up an RTSP stream https://github.com/sfalexrog/gst-rtsp-launch/
  • you will first need to figure out what gstreamer pipeline to use on voxl2 that will load your video and parse the h264/h265 frames (can use null sink for testing) and then use that pipeline with gst-rtsp-launch which will take the encoded frames and serve them over rtsp stream.
• gstreamer may be more flexible for tuning the encoding parameters of h264/h265 (compared to voxl-camera-server) and you can also use it in real time later (using voxl-streamer, which uses gstreamer under the hood)

Another alternative is to use voxl-record-raw-image to save raw YUVs coming from voxl-camera-server and then use voxl-replay and voxl-streamer - the latter will accept YUVs from the MPA pipe and encode them using the bit rate that you want. Note that depending on the image resolution, YUV images will take a lot more space than encoded video, but maybe that is also OK since VOXL2 has lots of storage.

Alex

Hello Rolando,

It will not be possible to fully use four ov7251 cameras at the same time due to i2c address conflict. the downward camera can coexist with the stereo pair because

• one of the stereo cameras is actually on a different i2c bus
• ov7251 camera has two i2c addresses, so with some software configuration it works. However, having 3 ov7251 cameras on i2c bus cannot be done without a conflict.

In a condition of i2c conflict between cameras, what will happen is..

• system will be able to detect each camera at boot time because probing happens while other cameras are powered off
• you should be able to fully use the conflicting cameras one at a time (not concurrently)
• if you use the conflicting cameras concurrently, commands sent to one camera will go out to both cameras, for example exposure and gain control. nothing will physically break, but this is currently not a supported use case.
• there will also be a short glitch after you started the first conflicting camera and when you are starting the second conflicting camera, since the first one will also get re-initialized with the commands intended for the second camera

in your case, the ov7251 plugged into J2 was not detected because the camera configuration xml file is not configured to query this camera on J2 connector. If you wanted to enable it, you need to back up and edit the following file : /system/etc/camera/camera_config.xml and add an entry that looks like below (then reboot). The slave address 0xC0 is one of the two possible slave addresses for this camera. The other one is 0xE0 and is already used by the camera plugged into J4 (downward facing).

So are you using one of the stereo cameras + stereo flex + extension cable? if you are using the stereo flex with one camera, make sure to use the left port on the flex because the right port is connected to the secondary i2c bus, accessible only in stereo configuration. Also, multiple stereo pairs are not supported on VOXL and stereo can only be used in J3.

With the below config addition, you should be able to detect all 4 ov7251 cameras and successfully use the following configurations without confilicts:

• J2 + J4 (hires slot + tracking slot)
• J3 + J4 (stereo slot + tracking slot)
• if you want to use J2 + J3 (hires + tracking slot), change the I2C id to 0xE0, but then it's the same thing as current stereo + tracking.

Now, in this configuration you should also be able to run all 4 cameras together but depending on the i2c ID of the camera in slot 0, it will conflict either with left stereo or tracking camera. I would not mess with stereo, so you could set i2c id of slot 0 to the same as in slot 2 (hires and tracking). then there is is still an issue of exposure/gain controls between the two cameras fighting with each other (both cameras will receive both exposure/gain updates).

In order to prevent two cameras from fighting exposure/gain settings the only way to do it is to prevent one of the camera drivers from sending out exposure/gain updates at all. Then both cameras will have the same exposure/gain computed based on one camera's image. This would require a little bit of work on the camera driver side but it's possible (though not supported now).

Anyway, this is just a long explanation of what you could try and what you will see.

<!-- ov7251 tracking camera in camera slot 0 -->
  <CameraModuleConfig>
    <CameraId>0</CameraId>
    <SensorName>ov7251</SensorName>
    <SensorSlaveAddress>0xC0</SensorSlaveAddress>
    <ChromatixName>ov7251_chromatix</ChromatixName>
    <ModesSupported>1</ModesSupported>
    <Position>FRONT</Position>
    <MountAngle>360</MountAngle>
    <CSIInfo>
      <CSIDCore>0</CSIDCore>
      <LaneMask>0x1F</LaneMask>
      <LaneAssign>0x4320</LaneAssign>
      <ComboMode>0</ComboMode>
    </CSIInfo>
    <LensInfo>
      <FocalLength>4.73</FocalLength>
      <FNumber>2.2</FNumber>
      <TotalFocusDistance>1.9</TotalFocusDistance>
      <HorizontalViewAngle>64.1</HorizontalViewAngle>
      <VerticalViewAngle>51.6</VerticalViewAngle>
      <MinFocusDistance>0.1</MinFocusDistance>
    </LensInfo>
  </CameraModuleConfig>

Hello @Gicu-Panaghiu,

I am going to assume you are using VOXL1, since you did not specify..

We do have RAW8 and RAW10 support for OV7251. The selection of the format has to be done in several places.

First, you have to select the correct camera driver, specifically..

ls /usr/lib/libmmcamera_ov7251*.so
/usr/lib/libmmcamera_ov7251.so
/usr/lib/libmmcamera_ov7251_8bit.so
/usr/lib/libmmcamera_ov7251_hflip_8bit.so
/usr/lib/libmmcamera_ov7251_rot180_8bit.so
/usr/lib/libmmcamera_ov7251_vflip_8bit.so

there are 5 options and one of them is _8bit.so which means it will natively ouptput 8bit data (all others output 10 bit data).

the driver name, such as ov7251_8bit has to be the sensor name <SensorName>ov7251_8bit</SensorName> in /system/etc/camera/camera_config.xml.

You can check camera_config.xml for what sensor library is used for your OV7251.

When you run voxl-configure-cameras script, it will actually copy one of the default camera_config.xml that are set up for a particular use case, and I believe it will indeed select the 8bit one - this was done to save cpu cycles needed to convert 10bit to 8bit, since majority of the time only 8bit pixels are used.

Now, you mentioned that HAL_PIXEL_FORMAT_RAW10 is passed to the stream config and unfortunately this does not have any effect on what the driver outputs. If the low level driver (e.g. libmmcamera_ov7251_8bit.so) is set up to output RAW8, it will output RAW8 if you request either HAL_PIXEL_FORMAT_RAW8 or HAL_PIXEL_FORMAT_RAW10.

So if you update the camera_config.xml to the 10bit driver and just keep the HAL_PIXEL_FORMAT_RAW10 in the stream config (then sync and reboot), you should be getting a 10 bit RAW image from the camera. But since the camera server is currently expecting 8 bit image, if you just interpret the image as 8 bit, it will appear garbled, so you will need to handle the 10 bit image (decide what you want to do with it) in the camera server.

Hello @david-moro ,

mvVISLAM library supports a mask file in the following format : 8 bits per pixel PGM with white (255) being where NO features should be found and black (0) where it is OK to have features. For a frame of aspect ratio 4:3, typically a mask of size 160x120 pixels should be used, although you can have a higher resolution mask if you want to. The mask will be resized to the actual image size or the smaller resolution mask will be properly queried (i do not know which one, but it should not matter). I would just suggest keeping the aspect ratio of the mask the same as the actual image.

You can test this easily by creating a mask that splits the image in half and seeing if features get detected in masked out region. Please note that the mask only affects detection of new features -- if the feature was detected in unmasked region and then the feature ends up in the masked region, it will still be tracked.

It does not look like a mask file is an option in voxl-qvio-server at the moment. You could try to add it yourself, here is an example how it was used in another project : https://github.com/ATLFlight/snap_vio/blob/master/src/snap_vio.cpp#L345C19-L345C19 . If mvVISLAM is not able to open the mask file you provided (wrong path, etc), it should output an error message like this : AR ERROR: arFileOpen(): Failed to open file: <path to mask file name>

I hope this helps!

Alex

The ipk is available here now : http://voxl-packages.modalai.com/stable/voxl-hal3-tof-cam-ros_0.0.5.ipk - you should be able to use the launch file to choose between two modes (5=short range and 9=long range) and fps, which are listed in the launch file.

@Samuel-Lehman , currently VOXL2 (M0154 variant, but not M0054 or VOXL2 mini M0104) has hardware support for display port output, the software is not available (no ETA at this point). VOXL2 is not intended to be used as a desktop computer, although it could be :).

However, if you just want to stream video without wifi / ethernet, you can actually use a cool trick : ADB port forwarding:

• plug in USBC cable from your VOXL2 in to linux pc
• run the following command to create a forwarding tunnel between voxl port 80 to port 8080 of your linux machine :

adb forward tcp:8080 tcp:80

If you have voxl-camera-server and voxl-portal already running, you should be able to view the camera streams in the browser at http://localhost:8080/video.html?cam=hires_color or something like that (depending on your camera name). I just tested it and it works well. You can use voxl-portal as you normally would, when using an network connection.

A few notes

• please make sure to use SDK 1.3.4 or later or at least voxl-portal because we recently added a fix for voxl-portal to work with port forwarding : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-portal/-/network/master?ref_type=heads
• voxl-portal can also display h264 video, but it needs a fix that i am working on and is almost done.

If this approach would work for you, you can test the non-encoded streams and we'll get the fix for h264 into voxl-portal soon.

Alex

@david-moro

If you look at this image, https://docs.modalai.com/M0169/#image-of-working-module it shows the TOF sensor (right) and the M0169 adapter (left).

The weight of the TOF module is 0.72g and weight of M0169 adapter is 0.93g.

This particular combo requires an external power input (4-5V as described here https://docs.modalai.com/M0169/#current-consumption). "External" in this case means additional power source besides the camera connector. The additional power input connects into M0169 via 2-pin JST connector.

The worse case power consumption is ~0.9W, which is a lot for such small module. The current draw plot shows a typical 5hz operation at maximum "exposure", which sends out 8 6-ms pulses of 1.0-1.1A (at 3.3V). so that is 1.1A * 3.3V * (5hz * 8 * 0.006s) = 0.87W. If you select 15Hz, the maximum exposure (emitter pulse time) will go down from 6ms to 2ms, so the maximum total power will remain the same.

This module would definitely require a heat spreader to operate continuously at this power. However maximum power output (even in auto exposure mode) can be limited to accommodate specific power dissipation capabilities. Additionally, the temperature of the emitter is reported from the TOF sensor.

Due to additional complexities that come with this new module (need for external power, which cannot be obtained from the camera connector) and heat management, we are working on the solution that would provide a positive user experience, so hence the delay.

I will check on the ETA

Hi

If you are looking for more low level functionality and a bit easier to experiment with, you may want to check out https://gitlab.com/voxl-public/utilities/voxl-rtsp/-/tree/dev . This tool is good for testing or running simple scenarios. This tool accepts some commands (like exposure control) via command line, so you could add a simple feature to save an image upon a command line input. Please see README to see how the exposure control input works and code for how it's implemented.

Please make sure that voxl-camera-server is disabled when you run voxl-rtsp.

If you still would like to go through the voxl-camera-server approach, we may need some more input from respective devs :).

I hope this helps..

Alex

@nekolny , you should use a linux PC to run voxl-esc tools in order to connect to the ESC and check / update the ESC firmware. VOXL1 does not have support for voxl-esc python tools to communicate directly with the ESC.

Alex

Hi @Kory4Drone ,

Starling 2 MAX uses two 2S Li-Ion batteries. When connected to the drone, they are connected in series, creating a single 4S battery. The - and + terminals of the combined battery are connected directly to the ESC.

You could tap off the power at the ESC main power pads. If the current for your aux functionality is low (up to a few Amps), then you could use the "Battery Input Test Points" on the opposite side from the main pads. These pads are via-stitched to the opposite side using many vias and should be solid.

https://docs.modalai.com/voxl-mini-esc-datasheet/#hardware-overview

I do not think we have the power wiring diagram for the drone posted.. will check.

Alex

@ROBERT-JUDD , the behavior of the RC receiver on the drone as is as follows:

• when the Transmitter (Commando 8 ) is off, the receiver does not send (broadcast) anything
• when the Transmitter is on, the receiver sends back telemetry packets (even if there are no telemetry updates from the PX4 side).

However, the GPS signal degradation seems to be related with proximity of the RC TX when the TX is on. The RC RX antenna is already located at the bottom of the drone and it should be OK, but there could also be some sort of interference, so possibly moving the RC RX antenna to the landing gear could resolve that (if it's an issue, TBD).

I don't personally have a spektrum analyzer for EMI scanning, but our team does and this work is also happening in parallel.

Will keep you posted!

Alex

Hello @Kiran , the only TOF sensor that we currently sell is the following:

https://www.modalai.com/products/m0178?variant=48528287793456

https://docs.modalai.com/M0178/

In terms of identifying unique features of individuals, you will have you decide whether this works for you. Here is a typical output of the this TOF sensor looking at a person. This is a depth image, but a point cloud output (for each frame) is also available. In order to build a map, you would need to use some mapper software (we provide a reference such as https://docs.modalai.com/voxl-mapper/). However you would need to be familiar with mapping algorithm(s) and learn how the software components work in order to get mapping to work well and tune it for your application.

Alex

@AidanGallagher , I have updated the IMX412 drivers to support stable 4040x3040 60FPS.

Just for full disclosure, the camera in this mode is slightly overclocked from 2.1Gbps / lane to 2.2Gbps MIPI bit rate (voxl2 supports up to 2.5Gbps) in order to allow sufficient time between EOF and SOF. This is within 5% of the camera spec and seems to work fine.. If you do not feel comfortable using this mode, don't :). (only the specific 4040x3040 60fps mode has been changed).

We are going to continue testing this mode because it allows doing things like EIS at 4K60 without losing any vertical stabilization margin. If there are any issues with it, I will report back.

By the way, for best image quality, even if you don't need 4K output, but lets say 1080p, in the case of unstabilized video (no EIS), you should select 3840x2160 input and 1920x1080 output. The debayering process loses sharpness, so going from 1920x1080 RAW -> 1920x1080 RGB is going to be worse quality than 3840x2160 RAW -> 1920x1080 RGB. For unstabilized video you do not need the full frame because the top and bottom will be cropped out anyway, so reduce the waste and not select the full frame resolution (4040x3040). You can experiment using the different input resolutions to see what output quality (sharpness) you get. There is currently no option to do output sharpening when using MISP, but we will look into in the future. The down side of using higher resolution input is increased GPU usage (should not be an issue) and longer readout time (the rolling shutter artifacts will be more pronounced).

https://storage.googleapis.com/modalai_public/temp/imx412_test_bins/20250409/imx412_fps_eis_20250409_drivers.zip

   - 4056x3040 @ 30                     16.5ms readout time
   - 4040x3040 @ 30, 60                 16.5ms readout time
   - 3840x2160 @ 30, 60, 80             11.8ms readout time
   - 1996x1520 @ 30, 60, 120            5.5ms readout time
   - 1936x1080 @ 30, 60, 90, 120, 240   4.0ms readout time
   - 1996x480  @ 30, 480                1.8ms readout time
   - 1996x240  @ 30, 800                0.9 - 1.0 ms readout time

@Dronodev , if you are using QVIO, which only supports a single camera, then you would address that use case by providing illumination for the single camera.

Our latest work with open-vins has shown results with triple camera vio using Starling 2 : https://www.youtube.com/watch?v=12DWDV7XCiE , so providing 360 degree illumination should help with that.

The images are normalized to help extract features in dark regions. However, illuminating the environment should help. Please keep in mind several points:

• adding light to the drone can result in shadows that move together with the vehicle. The shadows can be coming from drone components like landing gear, etc.
• you could use IR light and if the lens does not have the IR filter, the camera will see the IR light without it actually being visible to the human eye. The IR filter is typically installed inside the lens (right between the inner part of the lens before the light exits from the lens and goes into the camera sensor). For experimentation, the lens can be unscrewed from the camera and the IR filter carefully removed).

Our AR0144 tracking cameras have lens option with or without IR filter : https://www.modalai.com/products/msu-m0166

Alex

@AidanGallagher , the sync signal is used to synchronize multiple cameras to take the images at the same time - it is not related to the quality of the images. Some algorithms like DFS (Depth From Stere) require the images to be synchronized because the assumption of DFS requires that, otherwise disparity search can produce incorrect results.

For other applications like multi-camera VIO, it may or may not be required to use synchronized images -- unsync'ed camera images will have their own timestamps, which should be handled by the VIO algorithm.

The choice depends on the application.

Alex

@Morten-Nissov ,

Yes the old TOF (V1) is EOL and we only have the TOF V2 available now, but due to increase power requirements, the connector had to change. The Starling Front End board (M0173) provides the connector to plug in the M0178 tof assembly (look at https://docs.modalai.com/voxl2-coax-camera-bundles/ C26, or C27).

The Starling camera front end is only compatible with VOXL2, not VOXL2 mini. However, if you just wanted to plug in TOF V2 directly into VOX2 Mini, we now have M0172, which you can find a pictured here : https://forum.modalai.com/topic/4125/indoor-navigation-using-4-depth-sensors . M0172 splits the VOXL2 dual camera connector into a TOF-compatible 40-pin molex connector and the uCoax connector for the cameras that use uCoax.

We don't have M0172 officially available, but if you want to try it we can make it available in the shop. So you would need (M0172 adapter + MSU-M0178-1-01 (the whole TOF sensor assembly) ).

The connection order would look like this:
VOXL2 (mini) <-> M0172 <-> M0177 <-> M0178 <-> actual TOF sensor
(everything to the right of M0172 is sold in this kit : https://www.modalai.com/products/m0178?variant=48528287793456)

This would work for either VOXL2 or VOXL2 mini. The only down side i see is that M0177 is pretty short. I am not sure if we have an extension available, but will check.

EDIT: Here is the extension that can go between M0172 and M0177 : https://www.modalai.com/products/msu-m0170

Alex

@AidanGallagher , thanks for clarifying.

I will give you some background information that should help you understand what has happened before.

Since the camera itself can only output a single resolution and FPS at a time, the camera pipeline has to select one of several available modes / resolutions for the camera to operate. Since there are possibly multiple output streams when using Qualcomm ISP (yuv preview, small encoded, large encoded), there is some logic that looks at the resolutions of enabled streams and figures out what resolution should be picked for the requested FPS.

For example, if you request to streams : small encoded and large encoded, with resolutions 1280x720 and 1920x1080 respectively, and lets say fps is 30, the camera pipeline will try to pick resolution at least 1920x1080 in order to satisfy the largest requested output resolution. If the camera has both 1920x1080 and 3840x2160 available at 30fps, it will actually pick the larger one because the output quality will be a bit better and the ISP will do a good job downscaling from 4K to 1080p.

Now, if you then proceed to increase the FPS to 60, it is possible that the mode that was selected for 30fps is not actually available at 60FPS. Then the pipeline will try to find the best mode that the camera can provide, which can achieve 60FPS. It is sometimes the case that lower resolution / higher FPS modes are also cropped (in camera), so that is possibly why you were losing some FOV. If you needed to know exactly what is happening, then i need to know the specific resolutions and fps, etc and show you how to figur out which mode is selected, etc.

If you are using MISP and disable the Qualcomm ISP outputs, such as non-raw preview, small and large encoded, then the only stream that the camera pipeline is concerned about is the RAW stream, so the MISP input should be exactly the resolution and fps that you request using preview_width and preview_height. Then depending on the output of MISP, the MISP processing will downscale and crop (but crop only if needed). If the input and output are the same aspect ration, only down-scaling will be done.

I will also share with you that we recently added a feature to MISP that supports 3 additional output channels, individually enabled. In your IMX412 camera configuration, you can add the following entry (within the specific camera config) to enable up to 3 more channels with different resolutions, encoding type and bitrate. There are some limitations as the output channel fps cannot exceed the maximum fps specified by the top of the config for that camera.

So, together with the original misp output, this allows up to 4 output streams for the single camera (we kept the original misp output for backwards compatibility). The use case is that you can have different resolution streams and switch between them in real time (if there are no clients for the streams, they won't use cpu / gpu resources). The camera FPS is also adjusted based on the highest FPS of an active stream.

For each enabled output stream, you will have a new output "pipe" just like what you are already using.

Also when picking the output resolutions and fps, pay attention to the preview width and height you are selecting together with the FPS to make sure that is supported by the IMX412 driver. Also, MISP can automatically upscale to a larger resolution, but i don't know why you would want to do that.

This is an experimental feature, but feel free to check it out. the osd feature is not enabled, just a placeholder for now. You should use the latest camera server, which you can find in the dev package repository : http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64//

"misp_channels":  [
                {
                    "enable":       1,
                    "name":         "low_latency",
                    "width":        960,
                    "height":       540,
                    "fps":          120,
                    "mbps":         1.5,
                    "codec":        "h265",
                    "nPframes":     119,
                    "osd":          0
                },
                {
                    "enable":       1,
                    "name":         "long_range",
                    "width":        1280,
                    "height":       720,
                    "fps":          30,
                    "mbps":         2.5,
                    "codec":        "h265",
                    "nPframes":     29,
                    "osd":          0
                },
                {
                    "enable":       1,
                    "name":         "high_quality",
                    "width":        1920,
                    "height":       1080,
                    "fps":          60,
                    "mbps":         5.0,
                    "codec":        "h265",
                    "nPframes":     59,
                    "osd":          0
                }

@yardy , can you please clarify how exactly you are testing the communication using HS2 and HS3?