@dlee ,

Yes the new TOF sensor (IRS2975C) is more powerful that the previous generation. What I mean by that is that it can emit more IR power but also heats up more. Emitting more power allows the sensor detect objects at larger distances or objects that are not as reflective.

In current operating mode, the auto exposure control is enabled inside the sensor itself, which modulates the emitted IR power based on the returns that the sensor is getting. That is to say, the power draw will vary depending on what is in the view of the sensor. If there are obstacles nearby, the output power should be low, otherwise it can be high. At full power, the module can consume close to 0.8-0.9W

So the first solution, if design allows, is to add a heat spreader to dissipate the heat, which you already started experimenting with. The sensor has a large exposed copper pad in the back for heat sinking purposes for this exact reason. Just be careful not to short this pad to anything, use non-conducting (but heat transfering) adhesive pad between the sensor and heat spreader.

In terms of a software solution to the issue, we can query the temperature of the emitter. We can also control the maximum emitted power used by the auto exposure algorithm. That is to say, still leave the auto exposure running in the sensor, but limit the maximum power that it is allowed to use.

We are planning to add some software protection that limits the maximum output power as a function of the emitter temperature. This will require some implementation and testing.

Meanwhile, please consider using a heat spreader, which will be the best solution if you want to make use of the full sensor's operating range and not have our software limit the output power in order to prevent overheating.

Hello @Kashish-Garg-0

we have a curve that is "motor voltage vs rpm", meaning that for a desired RPM, it tells the ESC what average motor voltage should be applied. The average motor voltage is defined as battery_voltage * motor_pmw_duty_cycle. The battery voltage in this curve is in millivolts. Since you are typically controlling the desired RPM, as a user you do not need to worry about what "throttle" or voltage to apply - the ESC does this automatically in order to achieve the desired RPM. this calibration curve is used as a feed-forward term in the RPM controller. The ESC does support an "open loop" type of control where you specify the power from 0 to 100%, which is similar to a standard ESC, but PX4 does not use that ESC control mode.

By the way, you can test the ESC directly (not using PX4) using our voxl-esc tools (https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/master/voxl-esc-tools) which works directly on VOXL2 or a standalone linux PC (or mac). voxl-esc-spin.py has a --power argument where you specify the power from 0 to 100, which translates directly to the average duty cycle applied to the motor.

Here is the calibration for the Starling V2 motor / propeller that we use:
https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/mavic_mini_2/mavic_mini_2.xml?ref_type=heads#L63

Also, you can take a look at this post to see how to interpret those parameters a0, a1, a2 : https://forum.modalai.com/topic/2522/esc-calibration/2

We also have some dyno tests for this motor / propeller : https://gitlab.com/voxl-public/flight-core-px4/dyno_data/-/blob/master/data/mavic_mini2_timing_test/mavic_mini2_modal_esc_pusher_7.4V_timing0.csv . We are not sure how accurate that is, but it can be used as a starting point. @James-Strawson can you please confirm that is the correct dyno data for the Starling V2 motors?

Alex

@reber34 , perhaps this approach can work for you:

• record a video encoded at high bit rate (using voxl-camera-server and voxl-record-video . Please note that the output of voxl-record-video will not be in a standard container (such as mp4, etc), but you can fix it with ffpeg : ffmpeg -r 30 -i voxl-record-video.h264 -codec copy videofile.mp4
• re-encode the video offline with desired codecs / bit rates / resolutions
• install gst-rtsp-launch which uses gstreamer to set up an RTSP stream https://github.com/sfalexrog/gst-rtsp-launch/
  • you will first need to figure out what gstreamer pipeline to use on voxl2 that will load your video and parse the h264/h265 frames (can use null sink for testing) and then use that pipeline with gst-rtsp-launch which will take the encoded frames and serve them over rtsp stream.
• gstreamer may be more flexible for tuning the encoding parameters of h264/h265 (compared to voxl-camera-server) and you can also use it in real time later (using voxl-streamer, which uses gstreamer under the hood)

Another alternative is to use voxl-record-raw-image to save raw YUVs coming from voxl-camera-server and then use voxl-replay and voxl-streamer - the latter will accept YUVs from the MPA pipe and encode them using the bit rate that you want. Note that depending on the image resolution, YUV images will take a lot more space than encoded video, but maybe that is also OK since VOXL2 has lots of storage.

Alex

Hello Rolando,

It will not be possible to fully use four ov7251 cameras at the same time due to i2c address conflict. the downward camera can coexist with the stereo pair because

• one of the stereo cameras is actually on a different i2c bus
• ov7251 camera has two i2c addresses, so with some software configuration it works. However, having 3 ov7251 cameras on i2c bus cannot be done without a conflict.

In a condition of i2c conflict between cameras, what will happen is..

• system will be able to detect each camera at boot time because probing happens while other cameras are powered off
• you should be able to fully use the conflicting cameras one at a time (not concurrently)
• if you use the conflicting cameras concurrently, commands sent to one camera will go out to both cameras, for example exposure and gain control. nothing will physically break, but this is currently not a supported use case.
• there will also be a short glitch after you started the first conflicting camera and when you are starting the second conflicting camera, since the first one will also get re-initialized with the commands intended for the second camera

in your case, the ov7251 plugged into J2 was not detected because the camera configuration xml file is not configured to query this camera on J2 connector. If you wanted to enable it, you need to back up and edit the following file : /system/etc/camera/camera_config.xml and add an entry that looks like below (then reboot). The slave address 0xC0 is one of the two possible slave addresses for this camera. The other one is 0xE0 and is already used by the camera plugged into J4 (downward facing).

So are you using one of the stereo cameras + stereo flex + extension cable? if you are using the stereo flex with one camera, make sure to use the left port on the flex because the right port is connected to the secondary i2c bus, accessible only in stereo configuration. Also, multiple stereo pairs are not supported on VOXL and stereo can only be used in J3.

With the below config addition, you should be able to detect all 4 ov7251 cameras and successfully use the following configurations without confilicts:

• J2 + J4 (hires slot + tracking slot)
• J3 + J4 (stereo slot + tracking slot)
• if you want to use J2 + J3 (hires + tracking slot), change the I2C id to 0xE0, but then it's the same thing as current stereo + tracking.

Now, in this configuration you should also be able to run all 4 cameras together but depending on the i2c ID of the camera in slot 0, it will conflict either with left stereo or tracking camera. I would not mess with stereo, so you could set i2c id of slot 0 to the same as in slot 2 (hires and tracking). then there is is still an issue of exposure/gain controls between the two cameras fighting with each other (both cameras will receive both exposure/gain updates).

In order to prevent two cameras from fighting exposure/gain settings the only way to do it is to prevent one of the camera drivers from sending out exposure/gain updates at all. Then both cameras will have the same exposure/gain computed based on one camera's image. This would require a little bit of work on the camera driver side but it's possible (though not supported now).

Anyway, this is just a long explanation of what you could try and what you will see.

<!-- ov7251 tracking camera in camera slot 0 -->
  <CameraModuleConfig>
    <CameraId>0</CameraId>
    <SensorName>ov7251</SensorName>
    <SensorSlaveAddress>0xC0</SensorSlaveAddress>
    <ChromatixName>ov7251_chromatix</ChromatixName>
    <ModesSupported>1</ModesSupported>
    <Position>FRONT</Position>
    <MountAngle>360</MountAngle>
    <CSIInfo>
      <CSIDCore>0</CSIDCore>
      <LaneMask>0x1F</LaneMask>
      <LaneAssign>0x4320</LaneAssign>
      <ComboMode>0</ComboMode>
    </CSIInfo>
    <LensInfo>
      <FocalLength>4.73</FocalLength>
      <FNumber>2.2</FNumber>
      <TotalFocusDistance>1.9</TotalFocusDistance>
      <HorizontalViewAngle>64.1</HorizontalViewAngle>
      <VerticalViewAngle>51.6</VerticalViewAngle>
      <MinFocusDistance>0.1</MinFocusDistance>
    </LensInfo>
  </CameraModuleConfig>

Hello @Gicu-Panaghiu,

I am going to assume you are using VOXL1, since you did not specify..

We do have RAW8 and RAW10 support for OV7251. The selection of the format has to be done in several places.

First, you have to select the correct camera driver, specifically..

ls /usr/lib/libmmcamera_ov7251*.so
/usr/lib/libmmcamera_ov7251.so
/usr/lib/libmmcamera_ov7251_8bit.so
/usr/lib/libmmcamera_ov7251_hflip_8bit.so
/usr/lib/libmmcamera_ov7251_rot180_8bit.so
/usr/lib/libmmcamera_ov7251_vflip_8bit.so

there are 5 options and one of them is _8bit.so which means it will natively ouptput 8bit data (all others output 10 bit data).

the driver name, such as ov7251_8bit has to be the sensor name <SensorName>ov7251_8bit</SensorName> in /system/etc/camera/camera_config.xml.

You can check camera_config.xml for what sensor library is used for your OV7251.

When you run voxl-configure-cameras script, it will actually copy one of the default camera_config.xml that are set up for a particular use case, and I believe it will indeed select the 8bit one - this was done to save cpu cycles needed to convert 10bit to 8bit, since majority of the time only 8bit pixels are used.

Now, you mentioned that HAL_PIXEL_FORMAT_RAW10 is passed to the stream config and unfortunately this does not have any effect on what the driver outputs. If the low level driver (e.g. libmmcamera_ov7251_8bit.so) is set up to output RAW8, it will output RAW8 if you request either HAL_PIXEL_FORMAT_RAW8 or HAL_PIXEL_FORMAT_RAW10.

So if you update the camera_config.xml to the 10bit driver and just keep the HAL_PIXEL_FORMAT_RAW10 in the stream config (then sync and reboot), you should be getting a 10 bit RAW image from the camera. But since the camera server is currently expecting 8 bit image, if you just interpret the image as 8 bit, it will appear garbled, so you will need to handle the 10 bit image (decide what you want to do with it) in the camera server.

Hello @david-moro ,

mvVISLAM library supports a mask file in the following format : 8 bits per pixel PGM with white (255) being where NO features should be found and black (0) where it is OK to have features. For a frame of aspect ratio 4:3, typically a mask of size 160x120 pixels should be used, although you can have a higher resolution mask if you want to. The mask will be resized to the actual image size or the smaller resolution mask will be properly queried (i do not know which one, but it should not matter). I would just suggest keeping the aspect ratio of the mask the same as the actual image.

You can test this easily by creating a mask that splits the image in half and seeing if features get detected in masked out region. Please note that the mask only affects detection of new features -- if the feature was detected in unmasked region and then the feature ends up in the masked region, it will still be tracked.

It does not look like a mask file is an option in voxl-qvio-server at the moment. You could try to add it yourself, here is an example how it was used in another project : https://github.com/ATLFlight/snap_vio/blob/master/src/snap_vio.cpp#L345C19-L345C19 . If mvVISLAM is not able to open the mask file you provided (wrong path, etc), it should output an error message like this : AR ERROR: arFileOpen(): Failed to open file: <path to mask file name>

I hope this helps!

Alex

The ipk is available here now : http://voxl-packages.modalai.com/stable/voxl-hal3-tof-cam-ros_0.0.5.ipk - you should be able to use the launch file to choose between two modes (5=short range and 9=long range) and fps, which are listed in the launch file.

@Samuel-Lehman , currently VOXL2 (M0154 variant, but not M0054 or VOXL2 mini M0104) has hardware support for display port output, the software is not available (no ETA at this point). VOXL2 is not intended to be used as a desktop computer, although it could be :).

However, if you just want to stream video without wifi / ethernet, you can actually use a cool trick : ADB port forwarding:

• plug in USBC cable from your VOXL2 in to linux pc
• run the following command to create a forwarding tunnel between voxl port 80 to port 8080 of your linux machine :

adb forward tcp:8080 tcp:80

If you have voxl-camera-server and voxl-portal already running, you should be able to view the camera streams in the browser at http://localhost:8080/video.html?cam=hires_color or something like that (depending on your camera name). I just tested it and it works well. You can use voxl-portal as you normally would, when using an network connection.

A few notes

• please make sure to use SDK 1.3.4 or later or at least voxl-portal because we recently added a fix for voxl-portal to work with port forwarding : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-portal/-/network/master?ref_type=heads
• voxl-portal can also display h264 video, but it needs a fix that i am working on and is almost done.

If this approach would work for you, you can test the non-encoded streams and we'll get the fix for h264 into voxl-portal soon.

Alex

@david-moro

If you look at this image, https://docs.modalai.com/M0169/#image-of-working-module it shows the TOF sensor (right) and the M0169 adapter (left).

The weight of the TOF module is 0.72g and weight of M0169 adapter is 0.93g.

This particular combo requires an external power input (4-5V as described here https://docs.modalai.com/M0169/#current-consumption). "External" in this case means additional power source besides the camera connector. The additional power input connects into M0169 via 2-pin JST connector.

The worse case power consumption is ~0.9W, which is a lot for such small module. The current draw plot shows a typical 5hz operation at maximum "exposure", which sends out 8 6-ms pulses of 1.0-1.1A (at 3.3V). so that is 1.1A * 3.3V * (5hz * 8 * 0.006s) = 0.87W. If you select 15Hz, the maximum exposure (emitter pulse time) will go down from 6ms to 2ms, so the maximum total power will remain the same.

This module would definitely require a heat spreader to operate continuously at this power. However maximum power output (even in auto exposure mode) can be limited to accommodate specific power dissipation capabilities. Additionally, the temperature of the emitter is reported from the TOF sensor.

Due to additional complexities that come with this new module (need for external power, which cannot be obtained from the camera connector) and heat management, we are working on the solution that would provide a positive user experience, so hence the delay.

I will check on the ETA

Hi

If you are looking for more low level functionality and a bit easier to experiment with, you may want to check out https://gitlab.com/voxl-public/utilities/voxl-rtsp/-/tree/dev . This tool is good for testing or running simple scenarios. This tool accepts some commands (like exposure control) via command line, so you could add a simple feature to save an image upon a command line input. Please see README to see how the exposure control input works and code for how it's implemented.

Please make sure that voxl-camera-server is disabled when you run voxl-rtsp.

If you still would like to go through the voxl-camera-server approach, we may need some more input from respective devs :).

I hope this helps..

Alex

@scosgrove , I just wanted to mention that our ESC driver for PX4 is also using UART running at 2Mbit. The Flight Control loop sends out commands at 800Hz and each control packet is 15 bytes long. Also one ESC channel (out of four) sends feedback to the DSP for every command. The feedback packet size is 16 bytes (at 800hz), that would be 12800 bytes/s. We almost never see missed RX data. I have also experimented with larger feedback packets (~42 bytes at 800hz = 33600 bytes/s received by DSP).

Can you please clarify how the data is batched when the byte loss happens - how many bytes are sent without a break?

Alex

@Jainik-Mehta , can you please let me know what you are viewing in voxl-portal? A single camera (which) or all cameras?

Alex

@Jainik-Mehta ,

Thank you for providing the information. It seems VOXL2 is not actually idling. In different screen shots i see voxl-portal using some cpu and also voxl-camera-server using excessive amount of cpu.

Can you please provided some more details:

• is voxl2 enclosed or installed in to a drone or standalone? is it enclosed in a box?
• which VOXL2 SDK are you testing?
• what cameras are connected to VOXL2? Are you able to share your /etc/modalai/voxl-camera-server.conf ?
• how long does it take to for VOXL2 to overheat?

@Jetson-Nano , the following link shows location of all the connections and test points that are being utilized: https://docs.modalai.com/voxl-fpv-esc-datasheet/#hardware-overview . If there is a test point that is not documented, it means it is not supported in firmware or only used for production testing. If you have a specific question, please let me know.

The following link describes the Neopixel led support : https://docs.modalai.com/voxl-fpv-esc-datasheet/#neopixel-led-support . A single string with up to 32 LEDs is supported. VAUX regulator is dedicated for this output and can handle 500-600mA. We also have a way to control this output to be 3.3V or 5.0V (3.3V default) but the control feature is disabled at the moment. You can test the Neopixel output while PX4 is not running using voxl-esc tools : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-tools/voxl-esc-neopixel-test.py (we also have a helper command to use the LED while PX4 is running). https://gitlab.com/voxl-public/voxl-sdk/services/voxl-io-server/-/blob/master/tools/voxl-send-neopixel-cmd.c -- this tool is a command line tool which sends the control data to PX4 and then PX4 forwards it to the ESC. Right now the C example only supports white color, but we will extend it to RGB.

The payload connector just outputs VBATT, so as long as your desired current is within the JST connector spec, you should be ok. (4Amps per manufacturer's spec).

J1 is used for UART communication with FC or the voxl-esc test scripts and the data that is send are the ESC commands (rpm or power) and other commands like Neopixel commands, PWM output.. The ESC also sends back the ESC status.

Let us know if you have any more questions

Alex

@extmoddrv

We support the following modes for IMX412 cameras:

• Full frame 4056x3040 : 30FPS, possibly 60FPS (need to double check)
• 4K 60FPS
• 1080P 180FPS (maybe 240 FPS, need to double check)
• 720P (cropped) 240FPS

AR0144 FPS is currently limited to 50FPS (camera maximum is 60fps, but we have seen some stability issues). Resolution for AR0144 is fixed at 1280x800.

If you provide your specific requirements, I can confirm whether that is supported. Are you looking for the h264/5 encoded video for IMX412 camera? What about AR0144 tracking cameras - just RAW?

Alex

and also it would be helpful to see output of top and voxl-inspect-cpu just in case to have a more complete picture.

@jcai , please back up your camera configuration file for your reference and then modify it.

Since you only have one camera connected, please remove all the camera configs except for one imx412 and change the camera id for that camera to 0.

The camera configuration file is located in /etc/modalai/voxl-camera-server.conf. It is probably going to be easier to edit the file in a text editor, unless you are comfortable with vi / vim

If you want to add back other cameras, you first add the sensormodules back to /usr/lib/camera, then perform voxl-camera-server -l scan and see which cameras are detected as which IDs. Then update camera config file to add back the camera you need and double check the camera IDs that are listed in the camera config should match what you saw in voxl-camera-server -l. you can tell which camera is which by looking at Slave Address and Sensor Id output of voxl-camera-server -l.

For example the signature for IMX412 is:

Slave Address: 0x0034, Sensor Id: 0x0577

When camera server starts up, it goes through all the sensormodules in /usr/lib/camera and tries to use them to detect cameras on corresponding camera slots (each sensormodule is specific to a camera slot). Then the camera IDs are assigned the following way: lets say you detected three cameras in slots 1, 3, 4, so they will correspond to camera IDs (to be used in camera server config) as 0,1,2 (in the order of slot ids).

Alex

@mkriesel,

For global shutter camera like AR0144, the center of exposure will happen at the time that you suggested:

int64_t center_of_exposure_ns = start_of_exposure_ns + exposure_time_ns / 2;

For rolling shutter cameras, this is not true because different lines are exposed at different times. In this case, the center of exposure, which means the center of exposure of the middle line ( ) will be:

int64_t center_of_exposure_ns = start_of_exposure_ns + (exposure_time_ns + readout_time_ns)/2;

This means that you will need to know the readout_time (which is time it takes for the full image to be transferred from the camera). It is not currently exposed in camera server or the camera metadata.

Take a look at this resource, it should help understand the rolling shutter case if you need : https://www.1stvision.com/cameras/IDS/IDS-manuals/en/basics-shutter-mode.html , specifically:

Alex

@voxluser12341 , the qrb5165-io limitation only concerns the UARTs that are connected to the DSP, so SSC_QUP2, SSC_QUP6, SSC_QUP7 (dsp uarts 2,6,7).

The limitation does not apply to any Application Processor (APPS_QUP) ports which show up as /dev/ttyHSx

qrb5165-io library could be extended to support multiple concurrent ports, but we have not done that yet, unfortunately.

Alex

@voxluser12341 ,

Please note that currently qrb5165-io library only allows access to a single uart device, you cannot run it concurrently on multiple devices. Additionally, you cannot run qrb5165-io concurrently with PX4 (PX4 can access the uart ports concurrently)

The uart device numbers correspond to the QUP_SSCx number (where x is the uart bus number), you can review the pin definitions with pin names here : https://docs.modalai.com/voxl2-mini-connectors/#j19---external-sensors-2x-uart-2x-i2c. so, for example, J19 pins 7 and 8 are UART2 TX and RX.

The UART, I2C, SPI ports that are connected to the DSP (aka SSC, aka SLPI) are not exposed on the file system in /dev/

Alex