@dlee ,

Yes the new TOF sensor (IRS2975C) is more powerful that the previous generation. What I mean by that is that it can emit more IR power but also heats up more. Emitting more power allows the sensor detect objects at larger distances or objects that are not as reflective.

In current operating mode, the auto exposure control is enabled inside the sensor itself, which modulates the emitted IR power based on the returns that the sensor is getting. That is to say, the power draw will vary depending on what is in the view of the sensor. If there are obstacles nearby, the output power should be low, otherwise it can be high. At full power, the module can consume close to 0.8-0.9W

So the first solution, if design allows, is to add a heat spreader to dissipate the heat, which you already started experimenting with. The sensor has a large exposed copper pad in the back for heat sinking purposes for this exact reason. Just be careful not to short this pad to anything, use non-conducting (but heat transfering) adhesive pad between the sensor and heat spreader.

In terms of a software solution to the issue, we can query the temperature of the emitter. We can also control the maximum emitted power used by the auto exposure algorithm. That is to say, still leave the auto exposure running in the sensor, but limit the maximum power that it is allowed to use.

We are planning to add some software protection that limits the maximum output power as a function of the emitter temperature. This will require some implementation and testing.

Meanwhile, please consider using a heat spreader, which will be the best solution if you want to make use of the full sensor's operating range and not have our software limit the output power in order to prevent overheating.

Hello @Kashish-Garg-0

we have a curve that is "motor voltage vs rpm", meaning that for a desired RPM, it tells the ESC what average motor voltage should be applied. The average motor voltage is defined as battery_voltage * motor_pmw_duty_cycle. The battery voltage in this curve is in millivolts. Since you are typically controlling the desired RPM, as a user you do not need to worry about what "throttle" or voltage to apply - the ESC does this automatically in order to achieve the desired RPM. this calibration curve is used as a feed-forward term in the RPM controller. The ESC does support an "open loop" type of control where you specify the power from 0 to 100%, which is similar to a standard ESC, but PX4 does not use that ESC control mode.

By the way, you can test the ESC directly (not using PX4) using our voxl-esc tools (https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/master/voxl-esc-tools) which works directly on VOXL2 or a standalone linux PC (or mac). voxl-esc-spin.py has a --power argument where you specify the power from 0 to 100, which translates directly to the average duty cycle applied to the motor.

Here is the calibration for the Starling V2 motor / propeller that we use:
https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/mavic_mini_2/mavic_mini_2.xml?ref_type=heads#L63

Also, you can take a look at this post to see how to interpret those parameters a0, a1, a2 : https://forum.modalai.com/topic/2522/esc-calibration/2

We also have some dyno tests for this motor / propeller : https://gitlab.com/voxl-public/flight-core-px4/dyno_data/-/blob/master/data/mavic_mini2_timing_test/mavic_mini2_modal_esc_pusher_7.4V_timing0.csv . We are not sure how accurate that is, but it can be used as a starting point. @James-Strawson can you please confirm that is the correct dyno data for the Starling V2 motors?

Alex

@reber34 , perhaps this approach can work for you:

• record a video encoded at high bit rate (using voxl-camera-server and voxl-record-video . Please note that the output of voxl-record-video will not be in a standard container (such as mp4, etc), but you can fix it with ffpeg : ffmpeg -r 30 -i voxl-record-video.h264 -codec copy videofile.mp4
• re-encode the video offline with desired codecs / bit rates / resolutions
• install gst-rtsp-launch which uses gstreamer to set up an RTSP stream https://github.com/sfalexrog/gst-rtsp-launch/
  • you will first need to figure out what gstreamer pipeline to use on voxl2 that will load your video and parse the h264/h265 frames (can use null sink for testing) and then use that pipeline with gst-rtsp-launch which will take the encoded frames and serve them over rtsp stream.
• gstreamer may be more flexible for tuning the encoding parameters of h264/h265 (compared to voxl-camera-server) and you can also use it in real time later (using voxl-streamer, which uses gstreamer under the hood)

Another alternative is to use voxl-record-raw-image to save raw YUVs coming from voxl-camera-server and then use voxl-replay and voxl-streamer - the latter will accept YUVs from the MPA pipe and encode them using the bit rate that you want. Note that depending on the image resolution, YUV images will take a lot more space than encoded video, but maybe that is also OK since VOXL2 has lots of storage.

Alex

@smilon ,

I believe you are correct! Thank you. We will double check this and fix.

@Gary-Holmgren , you are correct, the high resolution camera is not used for VIO, we typically use it just for video recording / streaming.

You can certainly add a new transform to the extrinsics file and use it in your application. You should use the same name for the camera in the extrinsics file as you name it in voxl-camera-server.conf just to be consistent.

Hello @Gicu-Panaghiu,

I am going to assume you are using VOXL1, since you did not specify..

We do have RAW8 and RAW10 support for OV7251. The selection of the format has to be done in several places.

First, you have to select the correct camera driver, specifically..

ls /usr/lib/libmmcamera_ov7251*.so
/usr/lib/libmmcamera_ov7251.so
/usr/lib/libmmcamera_ov7251_8bit.so
/usr/lib/libmmcamera_ov7251_hflip_8bit.so
/usr/lib/libmmcamera_ov7251_rot180_8bit.so
/usr/lib/libmmcamera_ov7251_vflip_8bit.so

there are 5 options and one of them is _8bit.so which means it will natively ouptput 8bit data (all others output 10 bit data).

the driver name, such as ov7251_8bit has to be the sensor name <SensorName>ov7251_8bit</SensorName> in /system/etc/camera/camera_config.xml.

You can check camera_config.xml for what sensor library is used for your OV7251.

When you run voxl-configure-cameras script, it will actually copy one of the default camera_config.xml that are set up for a particular use case, and I believe it will indeed select the 8bit one - this was done to save cpu cycles needed to convert 10bit to 8bit, since majority of the time only 8bit pixels are used.

Now, you mentioned that HAL_PIXEL_FORMAT_RAW10 is passed to the stream config and unfortunately this does not have any effect on what the driver outputs. If the low level driver (e.g. libmmcamera_ov7251_8bit.so) is set up to output RAW8, it will output RAW8 if you request either HAL_PIXEL_FORMAT_RAW8 or HAL_PIXEL_FORMAT_RAW10.

So if you update the camera_config.xml to the 10bit driver and just keep the HAL_PIXEL_FORMAT_RAW10 in the stream config (then sync and reboot), you should be getting a 10 bit RAW image from the camera. But since the camera server is currently expecting 8 bit image, if you just interpret the image as 8 bit, it will appear garbled, so you will need to handle the 10 bit image (decide what you want to do with it) in the camera server.

Hello @david-moro ,

mvVISLAM library supports a mask file in the following format : 8 bits per pixel PGM with white (255) being where NO features should be found and black (0) where it is OK to have features. For a frame of aspect ratio 4:3, typically a mask of size 160x120 pixels should be used, although you can have a higher resolution mask if you want to. The mask will be resized to the actual image size or the smaller resolution mask will be properly queried (i do not know which one, but it should not matter). I would just suggest keeping the aspect ratio of the mask the same as the actual image.

You can test this easily by creating a mask that splits the image in half and seeing if features get detected in masked out region. Please note that the mask only affects detection of new features -- if the feature was detected in unmasked region and then the feature ends up in the masked region, it will still be tracked.

It does not look like a mask file is an option in voxl-qvio-server at the moment. You could try to add it yourself, here is an example how it was used in another project : https://github.com/ATLFlight/snap_vio/blob/master/src/snap_vio.cpp#L345C19-L345C19 . If mvVISLAM is not able to open the mask file you provided (wrong path, etc), it should output an error message like this : AR ERROR: arFileOpen(): Failed to open file: <path to mask file name>

I hope this helps!

Alex

The ipk is available here now : http://voxl-packages.modalai.com/stable/voxl-hal3-tof-cam-ros_0.0.5.ipk - you should be able to use the launch file to choose between two modes (5=short range and 9=long range) and fps, which are listed in the launch file.

@Prabhav-Gupta , yes it seems like OpenCV and ROS YUV_NV12 formats do not match up. I will take a look at it.. it seems the ROS YUV is packed (interleaved) while standard for storing YUV NV12 is having two planes : plane 1 : Y (size: widthheight), plane 2 : UV (size: widthheight/2)

In the meantime.. you can stream a rtsp h264/h265 from VOXL (use decent quality so that image looks good) and use opencv to receive the stream and get decompressed images: https://stackoverflow.com/questions/40875846/capturing-rtsp-camera-using-opencv-python

Would that work for you ? (unfortunately with rtsp stream, you will not get the full image metadata, like exposure, gain, timestamp, etc).

RTSP streaming can be done using voxl-streamer, which can accept either a YUV (and encode it) or already encoded h264/5 stream from voxl-camera-server.

Alex

@cfirth , if you are streaming raw RGB images, then 6404803 is 921KB, at 30fps will be 27MB/s which is definitely more than you want to transfer over wifi for a single stream. You can measure the available bandwidth using iperf as i suggested in previous post. Consider using compressed image format in ROS or use h264/5 encoded streams. You can use voxl-streamer to encode the tracking frames into a streaming video. If you describe your use case in more detail, we can provide additional suggestions.

@Sam-Kiley , yes, every Starling 2 / MAX is going to have M0173 board installed and needs to use the option 1.

Alex

@mkriesel , you should try to communicate with the ESC from a linux PC using a serial to usb adapter and voxl-esc tools. This will remove VOXL2 from the loop and allow you to test the ESC independently.

It seems there may be some intermittent issue, since you are saying that sometimes you are able to manually spin. what about voxl-esc-scan.py ? Check UART cable?

Alex

@psafi the html file is saved in the same folder where you run voxl-esc-spin* scripts if you have --enable-plot 1 . on VOXL2, the plot wont show up on screen, since there is no screen attached to VOXL2 (which would work on a linux pc), but it will be saved to html. In order to save to html, you will need to have plotly installed. You may first need to install pip3

apt update
apt install python3-pip
pip3 install plotly --upgrade

Alex

@Sam-Kiley ,

From your post i see that you are using the Linux Kernel that is configured for camera front end M0173. I can tell this from mach.var: 1.0.1 in your voxl-version output above.

You need to re-install your SDK and make sure to select the kernel option. You will need to select option 0. See the following thread for more details, however the that user's issue was the opposite - they had 1.0.0 and needed 1.0.1. You need 1.0.0.

https://forum.modalai.com/topic/4725/camera-not-working-after-upgrading-voxl-sdk/7

Alex

@psafi ,

We sometimes refer to VIO as QVIO because our VIO software uses a VIO library from Qualcomm (hence the Q). this implementation is robust and works well for a single camera.

For using multiple cameras, we are using a custom version of OpenVins.

Both voxl-qvio-server and voxl-open-vins-server are briefly mentioned in the debug section of the doc that you referred to : https://docs.modalai.com/flying-with-vio/#debugging

Alex

@psafi , it looks like the motor simply does not spin-up properly, most likely because the sinusoidal spin-up is enabled (which it is by default) and the motor parameters are not correct or there is not enough power to spin the motor during spinup.

for the sinusoidal spinup to work properly, you need to set the following parameters in the esc config file (i adjusted the parameters for your motor):

    <param name="vbat_nominal_mv"     value="22200"/>  <!-- used for sanity checking and limiting of voltage-dependent funcions -->
    <param name="num_cycles_per_rev"  value="7"/>      <!-- number of pole pairs in the motor. used for converting electrical frequency to mechanical rpm -->

    <param name="spinup_type"         value="1"/>      <!-- 0: traditional, 1: sinusoidal -->
    <param name="spinup_power"        value="120"/>    <!-- power used to during spin-up procedure -->
    <param name="latch_power"         value="120"/>    <!-- power used during latching stage of spin-up (out of 999)-->
    <param name="spinup_power_ramp"   value="16"/>     <!-- it will take ( 4096 / (spinup_power_ramp*10000) ) seconds to ramp sinusoidal start-up power from 0 to spinup_power -->
    <param name="spinup_rpm_target"   value="1500"/>   <!-- Desired RPM at the end of the sinusoidal spin-up procedure -->
    <param name="spinup_time_ms"      value="1500"/>   <!-- Duration of the sinusoidal spin-up procedure -->
    <param name="spinup_bemf_comp"    value="1"/>      <!-- 0: disable, 1:enable back-emf compensation in sinusoidal spin-up procedure -->
    <param name="motor_kv"            value="900"/>    <!-- kV value of the motor. used in back-emf compensation during spin-up -->

If the motor still struggles, you can try to increase the spin-up power a bit higher to 130 or 140, but double check the current (per motor) during spin-up - a good rule of thumb is not to exceed 2-3 amps per motor during the spin-up phase.

In the params above i set 1.5 seconds spin-up time. you can try shorter if needed. However, significantly shortening the spin-up period, may require slight increase in spin-up power (to give it more torque).

If you calibrate using ID2, you will also see the total current (which will also include power for VOXL2, but that should be minimal).

Also, you mentioned running calibration in several motors - you typically do not need to do that, but you certainly can, just to check for consistency. All the ESC channels should have the same params / calibration.

One more tip; you can test reliability of spin-up by automatically commanding the motor to start / stop using existing test script:

./voxl-esc-spin-step.py --id 2 --rpm 0 --step-amplitude 2000 --step-frequency 1  --timeout 10 --enable-plot 1 --cmd-rate 100

(please note that this is a RPM step command, so your motor should be calibrated for RPM control) . if you don't want to spin in rpm mode, you can just use the following command to do a power step:

./voxl-esc-spin-step.py --id 2 --power 0 --step-amplitude 10 --step-frequency 1  --timeout 10 --enable-plot 1 --cmd-rate 100

I ran a similar command on a much smaller motor and here is a result (the exact command i ran was the following, but it will be too fast for your big motor / prop. Additionally, i ran the test on a linux pc, which allowed me to get much higher rate of feedback (2000 hz)

./voxl-esc-spin-step.py --id 2 --rpm 0 --step-amplitude 3000 --step-frequency 3  --timeout 10 --enable-plot 1 --cmd-rate 2000 --step-delay 0.1

This type of test can be used to automate testing multiple start / stop. Since your motor is different, you should make sure the motor spins down completely before restarting (adjust --step-frequency).

Additional tip, if you want the motor to stop faster (at least for spin-up testing, or permanently), you can set the following param to 1 and the brake will be applied when the motor is commanded to stop spinning (not just coast down):

<param name="brake_to_stop"       value="1"/>             <!-- apply brake when stopping motor (or not) -->

Alex

@ysc , I am glad that the camera is working, but could you clarify what you changed to make it work? You should be able to get higher resolution and FPS if everything is working properly.

Also, if the logcat outout is not showing up, it is probably because the logging from camera pipeline is disabled in the following file : /vendor/etc/camera/camxoverridesettings.txt:

systemLogEnable=0

If that is set to zero, you can just comment out that parameter and try again.

Alex

@ysc ,

Thank you for performing the tests. It is possible that the cables were damaged, since they were working originally, it seems, even with the "fpv" version of the drivers.

Regarding the output resolution vs camera mode, the best way to find out which resolution the camera is actually streaming is to run the following before you start voxl-camera-server:

logcat | grep -i selected

The output should be something like this (may look slightly different in your case):

03-02 13:03:06.847  3338  3338 I CHIUSECASE: [CONFIG ] chxextensionmodule.cpp:2358 InitializeOverrideSession() Session_parameters FPS range 30:30, previewFPS 0, videoFPS 0 BatchSize: 1 HALOutputBufferCombined 0 FPS: 30 SkipPattern: 1, cameraId = 0 selected use case = 1
03-02 13:03:06.852  3338  3338 I CHIUSECASE: [CONFIG ] chxsensorselectmode.cpp:558 FindBestSensorMode() MAI: Selected Usecase: 7, SelectedMode W=4056, H=3040, FPS:30, NumBatchedFrames:0, modeIndex:0

this tells you that the camera mode 4056x3040 @ 30 FPS was selected from the several modes that the sensormodule driver provides. Since you have selected the ISP processing in your config file, the default logic in the camera pipeline will select highest resolution camera mode and then down-scale in the ISP (for highest quality). So, even though you were selecting 1280x720 output, it is most likely that the camera was still running in full resolution mode (you can check).

In any case, there is an issue with the flex cables. One last thing, I just wanted to confirm that you are NOT connecting two M0036 flexes back-to-back (to make a longer cable), because that is not guaranteed to work due to the length.

If the single M0036 cables are not working, then we will replace them for free. Please submit an RMA request (www.modalai.com/rma) and ask for QTY 2 free replacement M0036 cables (include a link to this thread)

Alex

@Federico-Wyrwal , thank you for the reply. More debugging would be needed to figure out the root cause and it looks like you already submitted an RMA for the inspection / repair, so we will take care of it. Thank you.

Alex

@ysc ,

the "fpv" version of the IMX412 driver uses the fastest camera MIPI bit rate at 2.1 Gbit/s per lane. The M0036 cables, when designed, were not tested under these conditions, since the camera driver update came recently and the test was not available.

Can you please try to use the IMX412 sensormodule from /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_2.bin instead of the FPV variant of the driver? (you will need to remove the com.qti.sensormodule.imx412_fpv_2.bin file from /usr/lib/camera/.

The older driver streams the camera data at a lower rate, I believe 1.5Gbit/s. Please let me know if this change works for you.

Regarding the cable supplier and degradation - we have not encountered cable degradation over time, however some cables may be slightly out of spec for this fastest MIPI bit rate. Also, we do not have other suppliers for the M0036, so you should purchase from ModalAI. However, lets figure out if there is a software solution first.

Also, what is the final resolution of the image and FPS that you need for your application? Lower resolutions require less bandwidth and we could lower the bitrate even more if that helps.

Alex