@dlee ,

Yes the new TOF sensor (IRS2975C) is more powerful that the previous generation. What I mean by that is that it can emit more IR power but also heats up more. Emitting more power allows the sensor detect objects at larger distances or objects that are not as reflective.

In current operating mode, the auto exposure control is enabled inside the sensor itself, which modulates the emitted IR power based on the returns that the sensor is getting. That is to say, the power draw will vary depending on what is in the view of the sensor. If there are obstacles nearby, the output power should be low, otherwise it can be high. At full power, the module can consume close to 0.8-0.9W

So the first solution, if design allows, is to add a heat spreader to dissipate the heat, which you already started experimenting with. The sensor has a large exposed copper pad in the back for heat sinking purposes for this exact reason. Just be careful not to short this pad to anything, use non-conducting (but heat transfering) adhesive pad between the sensor and heat spreader.

In terms of a software solution to the issue, we can query the temperature of the emitter. We can also control the maximum emitted power used by the auto exposure algorithm. That is to say, still leave the auto exposure running in the sensor, but limit the maximum power that it is allowed to use.

We are planning to add some software protection that limits the maximum output power as a function of the emitter temperature. This will require some implementation and testing.

Meanwhile, please consider using a heat spreader, which will be the best solution if you want to make use of the full sensor's operating range and not have our software limit the output power in order to prevent overheating.

Hello @Kashish-Garg-0

we have a curve that is "motor voltage vs rpm", meaning that for a desired RPM, it tells the ESC what average motor voltage should be applied. The average motor voltage is defined as battery_voltage * motor_pmw_duty_cycle. The battery voltage in this curve is in millivolts. Since you are typically controlling the desired RPM, as a user you do not need to worry about what "throttle" or voltage to apply - the ESC does this automatically in order to achieve the desired RPM. this calibration curve is used as a feed-forward term in the RPM controller. The ESC does support an "open loop" type of control where you specify the power from 0 to 100%, which is similar to a standard ESC, but PX4 does not use that ESC control mode.

By the way, you can test the ESC directly (not using PX4) using our voxl-esc tools (https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/master/voxl-esc-tools) which works directly on VOXL2 or a standalone linux PC (or mac). voxl-esc-spin.py has a --power argument where you specify the power from 0 to 100, which translates directly to the average duty cycle applied to the motor.

Here is the calibration for the Starling V2 motor / propeller that we use:
https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/mavic_mini_2/mavic_mini_2.xml?ref_type=heads#L63

Also, you can take a look at this post to see how to interpret those parameters a0, a1, a2 : https://forum.modalai.com/topic/2522/esc-calibration/2

We also have some dyno tests for this motor / propeller : https://gitlab.com/voxl-public/flight-core-px4/dyno_data/-/blob/master/data/mavic_mini2_timing_test/mavic_mini2_modal_esc_pusher_7.4V_timing0.csv . We are not sure how accurate that is, but it can be used as a starting point. @James-Strawson can you please confirm that is the correct dyno data for the Starling V2 motors?

Alex

@reber34 , perhaps this approach can work for you:

• record a video encoded at high bit rate (using voxl-camera-server and voxl-record-video . Please note that the output of voxl-record-video will not be in a standard container (such as mp4, etc), but you can fix it with ffpeg : ffmpeg -r 30 -i voxl-record-video.h264 -codec copy videofile.mp4
• re-encode the video offline with desired codecs / bit rates / resolutions
• install gst-rtsp-launch which uses gstreamer to set up an RTSP stream https://github.com/sfalexrog/gst-rtsp-launch/
  • you will first need to figure out what gstreamer pipeline to use on voxl2 that will load your video and parse the h264/h265 frames (can use null sink for testing) and then use that pipeline with gst-rtsp-launch which will take the encoded frames and serve them over rtsp stream.
• gstreamer may be more flexible for tuning the encoding parameters of h264/h265 (compared to voxl-camera-server) and you can also use it in real time later (using voxl-streamer, which uses gstreamer under the hood)

Another alternative is to use voxl-record-raw-image to save raw YUVs coming from voxl-camera-server and then use voxl-replay and voxl-streamer - the latter will accept YUVs from the MPA pipe and encode them using the bit rate that you want. Note that depending on the image resolution, YUV images will take a lot more space than encoded video, but maybe that is also OK since VOXL2 has lots of storage.

Alex

Hello Rolando,

It will not be possible to fully use four ov7251 cameras at the same time due to i2c address conflict. the downward camera can coexist with the stereo pair because

• one of the stereo cameras is actually on a different i2c bus
• ov7251 camera has two i2c addresses, so with some software configuration it works. However, having 3 ov7251 cameras on i2c bus cannot be done without a conflict.

In a condition of i2c conflict between cameras, what will happen is..

• system will be able to detect each camera at boot time because probing happens while other cameras are powered off
• you should be able to fully use the conflicting cameras one at a time (not concurrently)
• if you use the conflicting cameras concurrently, commands sent to one camera will go out to both cameras, for example exposure and gain control. nothing will physically break, but this is currently not a supported use case.
• there will also be a short glitch after you started the first conflicting camera and when you are starting the second conflicting camera, since the first one will also get re-initialized with the commands intended for the second camera

in your case, the ov7251 plugged into J2 was not detected because the camera configuration xml file is not configured to query this camera on J2 connector. If you wanted to enable it, you need to back up and edit the following file : /system/etc/camera/camera_config.xml and add an entry that looks like below (then reboot). The slave address 0xC0 is one of the two possible slave addresses for this camera. The other one is 0xE0 and is already used by the camera plugged into J4 (downward facing).

So are you using one of the stereo cameras + stereo flex + extension cable? if you are using the stereo flex with one camera, make sure to use the left port on the flex because the right port is connected to the secondary i2c bus, accessible only in stereo configuration. Also, multiple stereo pairs are not supported on VOXL and stereo can only be used in J3.

With the below config addition, you should be able to detect all 4 ov7251 cameras and successfully use the following configurations without confilicts:

• J2 + J4 (hires slot + tracking slot)
• J3 + J4 (stereo slot + tracking slot)
• if you want to use J2 + J3 (hires + tracking slot), change the I2C id to 0xE0, but then it's the same thing as current stereo + tracking.

Now, in this configuration you should also be able to run all 4 cameras together but depending on the i2c ID of the camera in slot 0, it will conflict either with left stereo or tracking camera. I would not mess with stereo, so you could set i2c id of slot 0 to the same as in slot 2 (hires and tracking). then there is is still an issue of exposure/gain controls between the two cameras fighting with each other (both cameras will receive both exposure/gain updates).

In order to prevent two cameras from fighting exposure/gain settings the only way to do it is to prevent one of the camera drivers from sending out exposure/gain updates at all. Then both cameras will have the same exposure/gain computed based on one camera's image. This would require a little bit of work on the camera driver side but it's possible (though not supported now).

Anyway, this is just a long explanation of what you could try and what you will see.

<!-- ov7251 tracking camera in camera slot 0 -->
  <CameraModuleConfig>
    <CameraId>0</CameraId>
    <SensorName>ov7251</SensorName>
    <SensorSlaveAddress>0xC0</SensorSlaveAddress>
    <ChromatixName>ov7251_chromatix</ChromatixName>
    <ModesSupported>1</ModesSupported>
    <Position>FRONT</Position>
    <MountAngle>360</MountAngle>
    <CSIInfo>
      <CSIDCore>0</CSIDCore>
      <LaneMask>0x1F</LaneMask>
      <LaneAssign>0x4320</LaneAssign>
      <ComboMode>0</ComboMode>
    </CSIInfo>
    <LensInfo>
      <FocalLength>4.73</FocalLength>
      <FNumber>2.2</FNumber>
      <TotalFocusDistance>1.9</TotalFocusDistance>
      <HorizontalViewAngle>64.1</HorizontalViewAngle>
      <VerticalViewAngle>51.6</VerticalViewAngle>
      <MinFocusDistance>0.1</MinFocusDistance>
    </LensInfo>
  </CameraModuleConfig>

Hello @Gicu-Panaghiu,

I am going to assume you are using VOXL1, since you did not specify..

We do have RAW8 and RAW10 support for OV7251. The selection of the format has to be done in several places.

First, you have to select the correct camera driver, specifically..

ls /usr/lib/libmmcamera_ov7251*.so
/usr/lib/libmmcamera_ov7251.so
/usr/lib/libmmcamera_ov7251_8bit.so
/usr/lib/libmmcamera_ov7251_hflip_8bit.so
/usr/lib/libmmcamera_ov7251_rot180_8bit.so
/usr/lib/libmmcamera_ov7251_vflip_8bit.so

there are 5 options and one of them is _8bit.so which means it will natively ouptput 8bit data (all others output 10 bit data).

the driver name, such as ov7251_8bit has to be the sensor name <SensorName>ov7251_8bit</SensorName> in /system/etc/camera/camera_config.xml.

You can check camera_config.xml for what sensor library is used for your OV7251.

When you run voxl-configure-cameras script, it will actually copy one of the default camera_config.xml that are set up for a particular use case, and I believe it will indeed select the 8bit one - this was done to save cpu cycles needed to convert 10bit to 8bit, since majority of the time only 8bit pixels are used.

Now, you mentioned that HAL_PIXEL_FORMAT_RAW10 is passed to the stream config and unfortunately this does not have any effect on what the driver outputs. If the low level driver (e.g. libmmcamera_ov7251_8bit.so) is set up to output RAW8, it will output RAW8 if you request either HAL_PIXEL_FORMAT_RAW8 or HAL_PIXEL_FORMAT_RAW10.

So if you update the camera_config.xml to the 10bit driver and just keep the HAL_PIXEL_FORMAT_RAW10 in the stream config (then sync and reboot), you should be getting a 10 bit RAW image from the camera. But since the camera server is currently expecting 8 bit image, if you just interpret the image as 8 bit, it will appear garbled, so you will need to handle the 10 bit image (decide what you want to do with it) in the camera server.

Hello @david-moro ,

mvVISLAM library supports a mask file in the following format : 8 bits per pixel PGM with white (255) being where NO features should be found and black (0) where it is OK to have features. For a frame of aspect ratio 4:3, typically a mask of size 160x120 pixels should be used, although you can have a higher resolution mask if you want to. The mask will be resized to the actual image size or the smaller resolution mask will be properly queried (i do not know which one, but it should not matter). I would just suggest keeping the aspect ratio of the mask the same as the actual image.

You can test this easily by creating a mask that splits the image in half and seeing if features get detected in masked out region. Please note that the mask only affects detection of new features -- if the feature was detected in unmasked region and then the feature ends up in the masked region, it will still be tracked.

It does not look like a mask file is an option in voxl-qvio-server at the moment. You could try to add it yourself, here is an example how it was used in another project : https://github.com/ATLFlight/snap_vio/blob/master/src/snap_vio.cpp#L345C19-L345C19 . If mvVISLAM is not able to open the mask file you provided (wrong path, etc), it should output an error message like this : AR ERROR: arFileOpen(): Failed to open file: <path to mask file name>

I hope this helps!

Alex

The ipk is available here now : http://voxl-packages.modalai.com/stable/voxl-hal3-tof-cam-ros_0.0.5.ipk - you should be able to use the launch file to choose between two modes (5=short range and 9=long range) and fps, which are listed in the launch file.

@Samuel-Lehman , currently VOXL2 (M0154 variant, but not M0054 or VOXL2 mini M0104) has hardware support for display port output, the software is not available (no ETA at this point). VOXL2 is not intended to be used as a desktop computer, although it could be :).

However, if you just want to stream video without wifi / ethernet, you can actually use a cool trick : ADB port forwarding:

• plug in USBC cable from your VOXL2 in to linux pc
• run the following command to create a forwarding tunnel between voxl port 80 to port 8080 of your linux machine :

adb forward tcp:8080 tcp:80

If you have voxl-camera-server and voxl-portal already running, you should be able to view the camera streams in the browser at http://localhost:8080/video.html?cam=hires_color or something like that (depending on your camera name). I just tested it and it works well. You can use voxl-portal as you normally would, when using an network connection.

A few notes

• please make sure to use SDK 1.3.4 or later or at least voxl-portal because we recently added a fix for voxl-portal to work with port forwarding : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-portal/-/network/master?ref_type=heads
• voxl-portal can also display h264 video, but it needs a fix that i am working on and is almost done.

If this approach would work for you, you can test the non-encoded streams and we'll get the fix for h264 into voxl-portal soon.

Alex

@david-moro

If you look at this image, https://docs.modalai.com/M0169/#image-of-working-module it shows the TOF sensor (right) and the M0169 adapter (left).

The weight of the TOF module is 0.72g and weight of M0169 adapter is 0.93g.

This particular combo requires an external power input (4-5V as described here https://docs.modalai.com/M0169/#current-consumption). "External" in this case means additional power source besides the camera connector. The additional power input connects into M0169 via 2-pin JST connector.

The worse case power consumption is ~0.9W, which is a lot for such small module. The current draw plot shows a typical 5hz operation at maximum "exposure", which sends out 8 6-ms pulses of 1.0-1.1A (at 3.3V). so that is 1.1A * 3.3V * (5hz * 8 * 0.006s) = 0.87W. If you select 15Hz, the maximum exposure (emitter pulse time) will go down from 6ms to 2ms, so the maximum total power will remain the same.

This module would definitely require a heat spreader to operate continuously at this power. However maximum power output (even in auto exposure mode) can be limited to accommodate specific power dissipation capabilities. Additionally, the temperature of the emitter is reported from the TOF sensor.

Due to additional complexities that come with this new module (need for external power, which cannot be obtained from the camera connector) and heat management, we are working on the solution that would provide a positive user experience, so hence the delay.

I will check on the ETA

Hi

If you are looking for more low level functionality and a bit easier to experiment with, you may want to check out https://gitlab.com/voxl-public/utilities/voxl-rtsp/-/tree/dev . This tool is good for testing or running simple scenarios. This tool accepts some commands (like exposure control) via command line, so you could add a simple feature to save an image upon a command line input. Please see README to see how the exposure control input works and code for how it's implemented.

Please make sure that voxl-camera-server is disabled when you run voxl-rtsp.

If you still would like to go through the voxl-camera-server approach, we may need some more input from respective devs :).

I hope this helps..

Alex

@jcai, yes you are correct regarding the connections that will work on Voxl2 and Voxl2 mini.

Please reach out to https://modalai.com/contact and we will create a custom order for you. You should add a link to this thread. I will work with our team to get the right hardware for you.

You will need

• M0159
• M0181
• two specific coax cables between those teo boards (18cm length?).
• M0155 (assuming you already have M0161 with a coax, but is your M0161 connected to M0173 camera front end?

You should also get a tool for removing coax connectors - check at the bottom of this guide: https://docs.modalai.com/micro-coax-user-guide/ . I dont think we sell the tool, but it looks like the tool is out of stock (Mouser) - you will need it to disconnect your M0161 from M0173.. i will check if we can provide it.

Regarding length, it looks like the longest cable we sell is 18cm : https://www.modalai.com/products/mcbl-00084?variant=48956821733680 , and this is what i am using for testing Hadron.

Alex

@jcai

Please see the following images of the Hadron 640R adapters. Please note that the hardware is subject to change, but this is our current version.

M0159 : Plugs into the back of Hadron. it has two micro coax cable coming out and an optional USB plug for connecting to the "Boson" inside Hadron

As I mentioned before, you can access the Boson camera via USB or MIPI.

M0181 is the adapter for connecting M0159 to VOXL2 / VOXL2 mini:

It is also possible to split up the two coaxes and instead of using a single M0181 for dual camera connection, M0155 for OV64B (https://www.modalai.com/products/m0155) and M0162 for Boson (or use USB for Boson).

@jcai ,

Yes, i will provide the photos of the boards.

And yes, it should work with M0173 + M0161 but only on voxl2. M0173 takes up two camera connectors J6 and J7, so the Hadron would connect to J8.

Another option is to use M0161 with just M0155 coax adapter, since connecting a single camera to M0173 is an overkill. M0155 + M0161 would only use one camera connector, allowing you to use M0161 and Hadron on voxl2 mini.

I will double check both options.

Alex

@Jetson-Nano , can you please clarify which connector on VOXL2 is your Flight Core connected to (for uart communication) and whether you have any indication that the communication is established? Have you previously been able to connect to VOXL2 from QGC and access the Flight Core V2 while it has been connected to VOXL2?

Alex

@Jetson-Nano , is voxl-mavlink-server actually running? you can check using voxl-inspect-services.

Also, are you sure that your Flight Core V2 is connected to /dev/ttyHS1 , which would be implied by using autopilot_uart_bus: 1 . How exactly are you connecting it to VOXL2?

Alex

@nshahrestani , at this time we do not have another version of the CAD model to share - can you please try to use another tool to convert it for your needs?

Alex

@AndriiHlyvko,

I just ran into the same issue while testing Boson (in my case I was testing the 16-bit Boson streaming mode). The libuvc issue caused a crash, since i did not use a fixed version of libuvc. However, what actually caused the buffer overflow was the fact that Boson was reporting smaller buffer requirements for RAW16 during query (it was reporting width * height * 1.5 , just like for YUV, but actual frame size was width * height * 2, so uvc pipeline allocated smaller buffer and overflow happened with the full frame came in.

Was this the same issue with your Boson tests or did you experience something else?

Thanks!

Alex

@tkddnjs825 , since you already have M0062, you should be able to use it. As you already identified, J9 on M0062 has /dev/ttyHS2 Linux UART device.

You can also find more information here : https://docs.modalai.com/voxl2-external-flight-controller/ regarding configuring mavlink server.

It looks like you are doing the right things here. However, the use case for "autopilot_uart_bus" maybe different from what you need - in your case your autopilot (px4) is actually on VOXL2 and the uart connection is for the path to GCS.

You may need to modify mavlink server depending on what you want to pass on this UART connection.. This goes beyond my knowledge of mavlink server - please try to see if you can get it to work, otherwise post a follow up question with details of what you would need to be sent using the UART connection.

Alex

@tkddnjs825 ,

I believe option 1 should work for you, since you will get a standard uart (linux device /dev/ttyHS1 which is completely independent from PX4 running on VOXL). However you will need the additional adapter board (either M0141 or M0151 - either one will work). You will need to configure voxl-mavlink-server to use this UART port to stream the messages that you need (you may already know how to do it, i am not the expert in this part).

By the way, if you use M0141 adapter board, you will also get USB 2.0 standard plug and if you use M0151 you will get USB 3.0 (make sure to get the version with cables). So you could plug in a wifi dongle or more advanced usb-to-wifi device (such as alfa networks awus036ach or similar) and not use Jetson if you do not need it.

If you plug in the wifi adapter directly into VOXL2 (via M0141 or M0151) then you will not need the additional UART (but you will have it anyway)

If you do not wish to get any additional hardware, you should be able to use option 3 to set up the TCP bridge between VOXL2 and Jetson, but that requires more knowledge of networking.

Alex

@Judoor-0 , can you please clarify what you mean a "flat image" ?

Also, it seems you can convert the type to MONO16 right in toCvCopy call. Take a look at the implementation, which will automatically re-scale the data (from MONO8 to MONO16), so you will not lose any information (because in this case you will be upscaling, that is going from 8 bit to 16 bit): https://github.com/strawlab/vision_opencv/blob/master/cv_bridge/src/cv_bridge.cpp#L174

So you can probably just have:

cv_bridge::CvImagePtr cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::MONO16);

// Publish the converted image
cv_bridge::CvImage out_msg;
out_msg.header = msg->header; // Keep the same header
out_msg.encoding = sensor_msgs::image_encodings::TYPE_16UC1;
out_msg.image = cv_ptr->image;
depth_pub_.publish(out_msg.toImageMsg());

or i think you can just do the following because toCvCopy will automatically copy the header and update the encoding for you:

cv_bridge::CvImagePtr cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::MONO16);
depth_pub_.publish(cv_ptr->toImageMsg());

I have not tried running it, but hopefully should point you in the right direction!

Alex