@dlee ,

Yes the new TOF sensor (IRS2975C) is more powerful that the previous generation. What I mean by that is that it can emit more IR power but also heats up more. Emitting more power allows the sensor detect objects at larger distances or objects that are not as reflective.

In current operating mode, the auto exposure control is enabled inside the sensor itself, which modulates the emitted IR power based on the returns that the sensor is getting. That is to say, the power draw will vary depending on what is in the view of the sensor. If there are obstacles nearby, the output power should be low, otherwise it can be high. At full power, the module can consume close to 0.8-0.9W

So the first solution, if design allows, is to add a heat spreader to dissipate the heat, which you already started experimenting with. The sensor has a large exposed copper pad in the back for heat sinking purposes for this exact reason. Just be careful not to short this pad to anything, use non-conducting (but heat transfering) adhesive pad between the sensor and heat spreader.

In terms of a software solution to the issue, we can query the temperature of the emitter. We can also control the maximum emitted power used by the auto exposure algorithm. That is to say, still leave the auto exposure running in the sensor, but limit the maximum power that it is allowed to use.

We are planning to add some software protection that limits the maximum output power as a function of the emitter temperature. This will require some implementation and testing.

Meanwhile, please consider using a heat spreader, which will be the best solution if you want to make use of the full sensor's operating range and not have our software limit the output power in order to prevent overheating.

Hello @Kashish-Garg-0

we have a curve that is "motor voltage vs rpm", meaning that for a desired RPM, it tells the ESC what average motor voltage should be applied. The average motor voltage is defined as battery_voltage * motor_pmw_duty_cycle. The battery voltage in this curve is in millivolts. Since you are typically controlling the desired RPM, as a user you do not need to worry about what "throttle" or voltage to apply - the ESC does this automatically in order to achieve the desired RPM. this calibration curve is used as a feed-forward term in the RPM controller. The ESC does support an "open loop" type of control where you specify the power from 0 to 100%, which is similar to a standard ESC, but PX4 does not use that ESC control mode.

By the way, you can test the ESC directly (not using PX4) using our voxl-esc tools (https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/master/voxl-esc-tools) which works directly on VOXL2 or a standalone linux PC (or mac). voxl-esc-spin.py has a --power argument where you specify the power from 0 to 100, which translates directly to the average duty cycle applied to the motor.

Here is the calibration for the Starling V2 motor / propeller that we use:
https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/mavic_mini_2/mavic_mini_2.xml?ref_type=heads#L63

Also, you can take a look at this post to see how to interpret those parameters a0, a1, a2 : https://forum.modalai.com/topic/2522/esc-calibration/2

We also have some dyno tests for this motor / propeller : https://gitlab.com/voxl-public/flight-core-px4/dyno_data/-/blob/master/data/mavic_mini2_timing_test/mavic_mini2_modal_esc_pusher_7.4V_timing0.csv . We are not sure how accurate that is, but it can be used as a starting point. @James-Strawson can you please confirm that is the correct dyno data for the Starling V2 motors?

Alex

@reber34 , perhaps this approach can work for you:

• record a video encoded at high bit rate (using voxl-camera-server and voxl-record-video . Please note that the output of voxl-record-video will not be in a standard container (such as mp4, etc), but you can fix it with ffpeg : ffmpeg -r 30 -i voxl-record-video.h264 -codec copy videofile.mp4
• re-encode the video offline with desired codecs / bit rates / resolutions
• install gst-rtsp-launch which uses gstreamer to set up an RTSP stream https://github.com/sfalexrog/gst-rtsp-launch/
  • you will first need to figure out what gstreamer pipeline to use on voxl2 that will load your video and parse the h264/h265 frames (can use null sink for testing) and then use that pipeline with gst-rtsp-launch which will take the encoded frames and serve them over rtsp stream.
• gstreamer may be more flexible for tuning the encoding parameters of h264/h265 (compared to voxl-camera-server) and you can also use it in real time later (using voxl-streamer, which uses gstreamer under the hood)

Another alternative is to use voxl-record-raw-image to save raw YUVs coming from voxl-camera-server and then use voxl-replay and voxl-streamer - the latter will accept YUVs from the MPA pipe and encode them using the bit rate that you want. Note that depending on the image resolution, YUV images will take a lot more space than encoded video, but maybe that is also OK since VOXL2 has lots of storage.

Alex

@vjuliani , the tracking camera ( i assume ov7251 or AR0144) is monochrome, so you can not make the image colored. You can install the latest version of the voxl-streamer from dev (nightly builds) http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64/ and it should allow you to encode h265.

Alex

@psafi , before calling voxl-gpio read, you need to enable the pin by setting it to input or output. Please check voxl-gpio -h. (hint: use voxl-gpio -m / voxl-gpio mode)

Alex

Hello @Gicu-Panaghiu,

I am going to assume you are using VOXL1, since you did not specify..

We do have RAW8 and RAW10 support for OV7251. The selection of the format has to be done in several places.

First, you have to select the correct camera driver, specifically..

ls /usr/lib/libmmcamera_ov7251*.so
/usr/lib/libmmcamera_ov7251.so
/usr/lib/libmmcamera_ov7251_8bit.so
/usr/lib/libmmcamera_ov7251_hflip_8bit.so
/usr/lib/libmmcamera_ov7251_rot180_8bit.so
/usr/lib/libmmcamera_ov7251_vflip_8bit.so

there are 5 options and one of them is _8bit.so which means it will natively ouptput 8bit data (all others output 10 bit data).

the driver name, such as ov7251_8bit has to be the sensor name <SensorName>ov7251_8bit</SensorName> in /system/etc/camera/camera_config.xml.

You can check camera_config.xml for what sensor library is used for your OV7251.

When you run voxl-configure-cameras script, it will actually copy one of the default camera_config.xml that are set up for a particular use case, and I believe it will indeed select the 8bit one - this was done to save cpu cycles needed to convert 10bit to 8bit, since majority of the time only 8bit pixels are used.

Now, you mentioned that HAL_PIXEL_FORMAT_RAW10 is passed to the stream config and unfortunately this does not have any effect on what the driver outputs. If the low level driver (e.g. libmmcamera_ov7251_8bit.so) is set up to output RAW8, it will output RAW8 if you request either HAL_PIXEL_FORMAT_RAW8 or HAL_PIXEL_FORMAT_RAW10.

So if you update the camera_config.xml to the 10bit driver and just keep the HAL_PIXEL_FORMAT_RAW10 in the stream config (then sync and reboot), you should be getting a 10 bit RAW image from the camera. But since the camera server is currently expecting 8 bit image, if you just interpret the image as 8 bit, it will appear garbled, so you will need to handle the 10 bit image (decide what you want to do with it) in the camera server.

It looks like your machine does not have QEMU support or is missing ARM support on your host machine. Please try instructions here : https://www.stereolabs.com/docs/docker/building-arm-container-on-x86/ (which also show you how to run ARM64 docker images on x86)

The ipk is available here now : http://voxl-packages.modalai.com/stable/voxl-hal3-tof-cam-ros_0.0.5.ipk - you should be able to use the launch file to choose between two modes (5=short range and 9=long range) and fps, which are listed in the launch file.

Hi

If you are looking for more low level functionality and a bit easier to experiment with, you may want to check out https://gitlab.com/voxl-public/utilities/voxl-rtsp/-/tree/dev . This tool is good for testing or running simple scenarios. This tool accepts some commands (like exposure control) via command line, so you could add a simple feature to save an image upon a command line input. Please see README to see how the exposure control input works and code for how it's implemented.

Please make sure that voxl-camera-server is disabled when you run voxl-rtsp.

If you still would like to go through the voxl-camera-server approach, we may need some more input from respective devs :).

I hope this helps..

Alex

@TomP , oh yeah, my bad.. you found the right one

@SKA ,

Does this happen right away after you start voxl-camera-server ?

What raw "preview" resolution / fps are you requesting in camera server config?

Also, can you post output of dmesg (everything after you start voxl-camera-server)

Alex

@SKA , that result / buffer error (i think) is showing up because the camera pipeline selects incorrect camera mode / resolution, perhaps you missed one of the steps in EIS instructions, either using the correct camera drivers or updating the override txt file.

Also, the latest docs for EIS are here, the old .md file is probably outdated : https://docs.modalai.com/camera-video/electronic-image-stabilization/

Alex

Hi @Rowan-Dempster ,

Please take a look at this example (you can build and run it too) : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/add-new-image-tools/tools/voxl-image-repub.cpp

This app can accept a regular image (RAW8 or YUV) and either re-publish it unchanged or crop and publish the result. Sometimes this is useful for quickly cropping an image that is fed into a different application that expects a smaller image or different aspect ratio.

The app shows how to subscribe and handle ion buffer streams.

Usage:

voxl2:/$ voxl-image-repub                          
ERROR: Pipe name not specified

    
    Re-publish cropped camera frames (RAW8 or YUV)
    
    Options are:
    -x, --crop-offset-x    crop offset in horizontal dimension
    -y, --crop-offset-y    crop offset in vertical dimension
    -w, --crop-size-x      crop size in horizontal dimension (width)
    -h, --crop-size-y      crop size in vertical dimension (height)
    -o, --output-name      output pipe name
    -u, --usage            print this help message
    
    The cropped image will be centered if the crop offsets are not provided. 
    
    typical usage:
    /# voxl-image-repub tracking --crop-size-x 256 --crop-size-y 256
    /# voxl-image-repub tracking --crop-size-x 256 --crop-size-y 256 --crop-offset-x 128 --crop-offset-y 128

example re-publishing ion buffer image as regular image (which you can view in voxl-portal ) :

voxl-image-repub tracking_front_misp_norm_ion -o test

(you can see which ion pipes are available by running voxl-list-pipes | grep _ion)

Please note that without the previous fix that i posted above, the client process that receives and uncached ION buffer will incur extra CPU load while accessing this buffer. For example, the same voxl-image-repub client uses 1.7% cpu while republishing the normalized image (cached ion buffer), while using 7.3% cpu republishing an image from an uncached ION buffer. (cpu usage % using one of the smaller cores).

Please try and let me know if you have any questions.

I know this cached / uncached buffering may be a bit confusing, but i will document this a bit more to help explain it a little better.

Alex

@SKA, you should be able to update libmodal-pipe to latest package in the dev package repository : http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64//

@alan123 , yes, the 12-pin connector on VOXL2 (J19) was designed such that the first 6 pins are compatible with a lot of existing GPS recevier modules. Please double check the pins and you should use a 12-pin connector to plug into VOXL2 (may need to re-pin your cable).

Alex

Hi @mark ,

Thank you for the update. We will keep investigating the underlying issue that was a result of that questionable camera.

Meanwhile, if you think that the camera is defective and has no damage, you can fill out an RMA request and send it back for an exchange. I cant really comment on what could be causing this on the camera itself (it would be interesting to try it), but the camera is not serviceable by the user.

https://www.modalai.com/pages/rma

Alex

Hi @Rowan-Dempster ,

I started a new branch where I will be working on some performance optimizations in the camera server.

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/tree/perf-optimizations

In my initial testing, setting cpu to perf and when running one or two instances of the following:

voxl-inspect-cam tracking_front_misp_norm tracking_down_misp_norm

i was seeing:

1 instance (2 inspected streams) : 42% CPU (of one core)
2 instances (4 inspected streams): 58% CPU (of one core)

with the changes i just committed, i am seeing:

1 instance: 31% cpu
2 instances : 36% cpu

If you would like you can test camera server from this branch and see if you can reproduce the results.

notes:

• the internal buffers were switched from uncached to cached and proper buffer management was added to ensure that data written by GPU is properly accessed by CPU
• with these changes, if you use the _encoded stream from the tracking camera, it will work, but in dmesg you will see messages related to qbuf cache ops failed -- this is still under investigation and will be fixed soon.

Meanwhile, I will work on an a simple example that shows the usage of ION buffers, I will try to share it a bit later today.

Alex

tagging @Eric-Katzfey for Ardupiot -related question..

Some answers:
3-1: VOXL ESC supports PWM input via aux pads (as you probably already know, but just noting it here) : https://docs.modalai.com/voxl-mini-esc-datasheet/#pwm-inputs--outputs. The range of the commands are 1000us=0% duty cycle, 2000us = 100% duty cycle (linear mapping). There is no calibration needed or supported (the MCU clock is pretty accurate).

3-2: Recommended PWM frequency (up to 1/(2000us) = 500hz as long as pulses have some off time between them), one-shot is supported.

3-3: i don't think that we have tested the mini ESC with Ardupilot in PWM mode, but the ESC should work very similarly to a standard ESC in PWM control mode

3-4: there is no way to set the zero-throttle PWM value in the ESC params, however here is my recommendation:

• in order to stop the motor and keep it off, send pulse 950us
• in order to start the motor, send pulse 1050us and operate in range 1050-2000us.
• the turn off point is 1000us with some hysteresis, but you don't want to operate close to that point and risk motors shutting off in flight.

Also, we do not support Dshot input for VOXL ESCs. Spin direction reversal is not supported with the PWM commands.

UART protocol is recommended because you don't run into the PWM-related range issues and you also get telemetry back from the ESCs (and much higher command rate is supported).

Alex

@joseph-vale ,

You did not provide the actual error that you are seeing, however I could try to guess what it is (even if not, the details below should probably help you anyway). The default build of voxl-opencv package does not have python3 support. So if you are using gstreamer with opencv in python and that is the error, you should install the voxl-opencv package that I built with python3 support

• https://storage.googleapis.com/modalai_public/temp/voxl-opencv_4.5.5-3_arm64.deb
• source : https://gitlab.com/voxl-public/voxl-sdk/third-party/voxl-opencv/-/tree/add-python3-bindings

Below, i will assume that you want to grab images from Boson part of the Hadron, however similar approach should apply to the RGB camera in Hadron.

First you should test ability to grab images without python. You may need to replace your camera # depending on what camera id your Boson is.

IMPORTANT: make sure that voxl-camear-server is not running while you are trying to use gstreamer.

systemctl stop voxl-camera-server

Tip: you can actually stream video using X forwarding with ssh. This should stream live Boson feed from voxl2 to your linux machine:

ssh -Y username@<voxl-ip>
gst-launch-1.0 qtiqmmfsrc camera=0 ! "video/x-raw,width=640,height=512,framerate=30/1" ! videoconvert ! autovideosink

display image directly in terminal as ascii:

gst-launch-1.0 qtiqmmfsrc camera=0 ! "video/x-raw,width=640,height=512,framerate=30/1" ! autovideoconvert ! aasink

and then finally, a python script that grabs h264 video from qtiqmmfsrc, decodes it and returns frames to python:

import time
import cv2

#get RGB (BGR?) directly
#stream='gst-launch-1.0 qtiqmmfsrc camera=0 ! video/x-raw, width=640,height=512,framerate=30/1 ! autovideoconvert ! appsink'

#get h264 -> decode -> BGR
stream='gst-launch-1.0 qtiqmmfsrc camera=0 ! video/x-h264,format=NV12,profile=high,width=640,height=512,framerate=30/1 ! h264parse ! qtivdec ! qtivtransform ! video/x-raw,format=BGR,width=640,height=512 ! autovideoconvert ! appsink'

print(stream)
vcap = cv2.VideoCapture(stream,cv2.CAP_GSTREAMER)

frame_cntr = 0
while(1):
    ret, frame = vcap.read()
    if ret:
        frame_cntr += 1
        print('got frame %d with dims ' % frame_cntr, frame.shape)

Hopefully, that works for you.

Final recommendation - if you use qtiqmmfsrc this way, the Boson data is processed in the Qualcomm ISP and unless your have a special tuning file for Boson, the processed output will have degraded quality. Boson, by default, outputs post AGC 8-bit image which is already processed and does not need to be further processed by the ISP. I am not sure whether you can get RAW8 data from qtiqmmfsrc (unmodified data from Boson).

We handle the above issue in voxl-camera-server by working with the RAW8 directly. We also recently started experimenting with 14bit pre-AGC data from Boson, which would need some processing before it is usable (if you are interested in that, i can share some more information).

Finally, if you would like to use voxl-camera-server, which is what we recommend and support, there is also a way to get encoded h264/h265 data into python (using our experimental pympa (python MPA bindings)). That is a topic for a discussion in another post, if you are interested..

Alex

@Rowan-Dempster , you should use a monochrome stream (_grey), since QVIO needs a RAW8 image.

If you are not using MISP on hires cameras, that is fine, you can start off using the output of the ISP.

You should calibrate the camera using whatever resolution you decide to try. This is to avoid any confusion, since if you using ISP pipeline, the camera pipeline may select a higher resolution and downscale + crop. So whenever you are changing resolutions, it is always good to do a quick camera calibration to confirm the camera parameters.

When using MISP, we have more control over which camera mode is selected, because MISP gets the RAW data, not processed by the ISP, so we know the exact dimensions of the image sent from camera.

Alex