Best posts made by Alex Kushleyev

@dlee ,

Yes the new TOF sensor (IRS2975C) is more powerful that the previous generation. What I mean by that is that it can emit more IR power but also heats up more. Emitting more power allows the sensor detect objects at larger distances or objects that are not as reflective.

In current operating mode, the auto exposure control is enabled inside the sensor itself, which modulates the emitted IR power based on the returns that the sensor is getting. That is to say, the power draw will vary depending on what is in the view of the sensor. If there are obstacles nearby, the output power should be low, otherwise it can be high. At full power, the module can consume close to 0.8-0.9W

So the first solution, if design allows, is to add a heat spreader to dissipate the heat, which you already started experimenting with. The sensor has a large exposed copper pad in the back for heat sinking purposes for this exact reason. Just be careful not to short this pad to anything, use non-conducting (but heat transfering) adhesive pad between the sensor and heat spreader.

In terms of a software solution to the issue, we can query the temperature of the emitter. We can also control the maximum emitted power used by the auto exposure algorithm. That is to say, still leave the auto exposure running in the sensor, but limit the maximum power that it is allowed to use.

We are planning to add some software protection that limits the maximum output power as a function of the emitter temperature. This will require some implementation and testing.

Meanwhile, please consider using a heat spreader, which will be the best solution if you want to make use of the full sensor's operating range and not have our software limit the output power in order to prevent overheating.

Hello @Kashish-Garg-0

we have a curve that is "motor voltage vs rpm", meaning that for a desired RPM, it tells the ESC what average motor voltage should be applied. The average motor voltage is defined as battery_voltage * motor_pmw_duty_cycle. The battery voltage in this curve is in millivolts. Since you are typically controlling the desired RPM, as a user you do not need to worry about what "throttle" or voltage to apply - the ESC does this automatically in order to achieve the desired RPM. this calibration curve is used as a feed-forward term in the RPM controller. The ESC does support an "open loop" type of control where you specify the power from 0 to 100%, which is similar to a standard ESC, but PX4 does not use that ESC control mode.

By the way, you can test the ESC directly (not using PX4) using our voxl-esc tools (https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/tree/master/voxl-esc-tools) which works directly on VOXL2 or a standalone linux PC (or mac). voxl-esc-spin.py has a --power argument where you specify the power from 0 to 100, which translates directly to the average duty cycle applied to the motor.

Here is the calibration for the Starling V2 motor / propeller that we use:
https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/mavic_mini_2/mavic_mini_2.xml?ref_type=heads#L63

Also, you can take a look at this post to see how to interpret those parameters a0, a1, a2 : https://forum.modalai.com/topic/2522/esc-calibration/2

We also have some dyno tests for this motor / propeller : https://gitlab.com/voxl-public/flight-core-px4/dyno_data/-/blob/master/data/mavic_mini2_timing_test/mavic_mini2_modal_esc_pusher_7.4V_timing0.csv . We are not sure how accurate that is, but it can be used as a starting point. @James-Strawson can you please confirm that is the correct dyno data for the Starling V2 motors?

Alex

@reber34 , perhaps this approach can work for you:

• record a video encoded at high bit rate (using voxl-camera-server and voxl-record-video . Please note that the output of voxl-record-video will not be in a standard container (such as mp4, etc), but you can fix it with ffpeg : ffmpeg -r 30 -i voxl-record-video.h264 -codec copy videofile.mp4
• re-encode the video offline with desired codecs / bit rates / resolutions
• install gst-rtsp-launch which uses gstreamer to set up an RTSP stream https://github.com/sfalexrog/gst-rtsp-launch/
  • you will first need to figure out what gstreamer pipeline to use on voxl2 that will load your video and parse the h264/h265 frames (can use null sink for testing) and then use that pipeline with gst-rtsp-launch which will take the encoded frames and serve them over rtsp stream.
• gstreamer may be more flexible for tuning the encoding parameters of h264/h265 (compared to voxl-camera-server) and you can also use it in real time later (using voxl-streamer, which uses gstreamer under the hood)

Another alternative is to use voxl-record-raw-image to save raw YUVs coming from voxl-camera-server and then use voxl-replay and voxl-streamer - the latter will accept YUVs from the MPA pipe and encode them using the bit rate that you want. Note that depending on the image resolution, YUV images will take a lot more space than encoded video, but maybe that is also OK since VOXL2 has lots of storage.

Alex

@smilon , voxl-esc-calibrate.py is a script that runs a test procedure in a single motor (with propeller mounted) to calibrate the behavior of the motor / propeller. This procedure only needs to be run once if you change motor or propeller type from a default configuration. The output of this script is just 3 coeficients a1, a2, a3 which you would need to manually enter into an ESC calibration xml file and then upload the xml paramer file to the ESC. Full details about the ESC calibration (when to do it and how) can be found here : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-tools/calibration.md?ref_type=heads

If you are using standard motors and propellers (one of standard ModalAI drones), you do not need to run this calibration procedure.

It sounds like you got it working, I believe voxl-configre-mpa took care of it. You can see what voxl-configure-mpa typically does here : https://docs.modalai.com/voxl-configure-mpa/ , which includes running voxl-esc to upload the latest firmware and params for a specific vehicle.

@wilkinsaf , M0027 was never shipped to any customers, as it was a test version of the ESC. So there should only be M0049, M0117, M0134 and M0129 (mini) ESCs out there. Like Vinny said, all of those ESCs have a blue status LED for each MCU.

If your ESC has a larger rectangular shape (as opposed to a square), it could be a really old ESC (Atmel-based, not STM32-based), which we do not really support any more. I hope this helps!

Alex

Hello @Gicu-Panaghiu,

I am going to assume you are using VOXL1, since you did not specify..

We do have RAW8 and RAW10 support for OV7251. The selection of the format has to be done in several places.

First, you have to select the correct camera driver, specifically..

ls /usr/lib/libmmcamera_ov7251*.so
/usr/lib/libmmcamera_ov7251.so
/usr/lib/libmmcamera_ov7251_8bit.so
/usr/lib/libmmcamera_ov7251_hflip_8bit.so
/usr/lib/libmmcamera_ov7251_rot180_8bit.so
/usr/lib/libmmcamera_ov7251_vflip_8bit.so

there are 5 options and one of them is _8bit.so which means it will natively ouptput 8bit data (all others output 10 bit data).

the driver name, such as ov7251_8bit has to be the sensor name <SensorName>ov7251_8bit</SensorName> in /system/etc/camera/camera_config.xml.

You can check camera_config.xml for what sensor library is used for your OV7251.

When you run voxl-configure-cameras script, it will actually copy one of the default camera_config.xml that are set up for a particular use case, and I believe it will indeed select the 8bit one - this was done to save cpu cycles needed to convert 10bit to 8bit, since majority of the time only 8bit pixels are used.

Now, you mentioned that HAL_PIXEL_FORMAT_RAW10 is passed to the stream config and unfortunately this does not have any effect on what the driver outputs. If the low level driver (e.g. libmmcamera_ov7251_8bit.so) is set up to output RAW8, it will output RAW8 if you request either HAL_PIXEL_FORMAT_RAW8 or HAL_PIXEL_FORMAT_RAW10.

So if you update the camera_config.xml to the 10bit driver and just keep the HAL_PIXEL_FORMAT_RAW10 in the stream config (then sync and reboot), you should be getting a 10 bit RAW image from the camera. But since the camera server is currently expecting 8 bit image, if you just interpret the image as 8 bit, it will appear garbled, so you will need to handle the 10 bit image (decide what you want to do with it) in the camera server.

The ipk is available here now : http://voxl-packages.modalai.com/stable/voxl-hal3-tof-cam-ros_0.0.5.ipk - you should be able to use the launch file to choose between two modes (5=short range and 9=long range) and fps, which are listed in the launch file.

@Moderator said in VOXL ESC Mini 4-in-1 Current per Motor:

Is it possible to step up voltage?

Can you please clarify the question?

Mini ESC is designed for small drones ( < 500g ). The ESC has been tested to handle 15A continous at 15V input continuously (60+ seconds), but with full direct air flow from propellers. This would simulate a full throttle "punch-out" on a small FPV drone (high current, but also lots of direct airflow = cooling). Do not use this ESC if the drone needs 10-15A per channel just to hover. Use it in application where hover current per motor is less than 5A (ideally 2-3A which is very typical) and absolute maximum continuous current per motor can be 10-15A.

For example, motors used for small FPV drones often are around 1306 size (3-4S Lipo). Those motors are usually rated for up to 10-12A continous (for 30-60 seconds). Larger motors can be used as long as maximum motor current does not exceed 10-15A (still 2-3A at hover) and there is sufficient cooling.

Always check ESC board temperature during initial flights / tuning. Temperature must stay below 110C at all times (critical), typically in the range of 40-70C for most applications. The ESC will most likely fail above 125C.

Temperature of the ESC board is the limiting factor because the board is so small. Mosfets can handle a lot of current as long as they don't overheat. So the design of the drone is very important (either use low current so that temperature is not an issue or properly design air flow from propellers and/or add heat spreader to keep the ESC board temperature in normal range for higher current draw applications).

ESC provides real time temperature feedback and it can be viewed in PX4 / QGC. Additionally, the PX4 logs contain the temperature information.

@John-Nomikos-0 , the ESC params / settings are completely separate from PX4. PX4 cannot modify the ESC params - all the ESC configuration is done through our tool voxl-esc. You should make sure the correct ESC params are installed before starting PX4.

PX4 just has the interface to control the ESC by sending the RPM commands to spin the motors.

Alex

@david-moro

If you look at this image, https://docs.modalai.com/M0169/#image-of-working-module it shows the TOF sensor (right) and the M0169 adapter (left).

The weight of the TOF module is 0.72g and weight of M0169 adapter is 0.93g.

This particular combo requires an external power input (4-5V as described here https://docs.modalai.com/M0169/#current-consumption). "External" in this case means additional power source besides the camera connector. The additional power input connects into M0169 via 2-pin JST connector.

The worse case power consumption is ~0.9W, which is a lot for such small module. The current draw plot shows a typical 5hz operation at maximum "exposure", which sends out 8 6-ms pulses of 1.0-1.1A (at 3.3V). so that is 1.1A * 3.3V * (5hz * 8 * 0.006s) = 0.87W. If you select 15Hz, the maximum exposure (emitter pulse time) will go down from 6ms to 2ms, so the maximum total power will remain the same.

This module would definitely require a heat spreader to operate continuously at this power. However maximum power output (even in auto exposure mode) can be limited to accommodate specific power dissipation capabilities. Additionally, the temperature of the emitter is reported from the TOF sensor.

Due to additional complexities that come with this new module (need for external power, which cannot be obtained from the camera connector) and heat management, we are working on the solution that would provide a positive user experience, so hence the delay.

I will check on the ETA

Hi Ryan,

Does the micrortps agent need to open a standard UART port, such as /dev/ttySx or /dev/ttyUSBx or similar? how hard is it to extend the micrortps client to use libvoxl_io functionality to access the serial port via libvoxl_io instead of standard serial port libraries? You would need to use this header https://gitlab.com/voxl-public/core-libs/libvoxl_io/-/blob/master/lib/include/voxl_io.h and link against libvoxl_io.

If changing micrortps code is not an option, then some sort of bridge could work. I have done similar things before using pseudo terminals (pty/tty pairs). Basically the bridge could be the master and would read/write using ptyXY and micrortps would be the client and read/write using ttyXY. However, i don't think pseudo terminals are available on VOXL.

https://stackoverflow.com/questions/52187/virtual-serial-port-for-linux this discusses the pseudo terminal as well another solution using socat. Now, socat is not available on VOXL either, but if you use docker, it can be installed in a small container using apt-get install socat.

Let me check if we can easily add the socat and pseudo terminals to the VOXL system image..

It seems perhaps the most robust solution (for now) would be extending micrortps to work with libvolx_io serial ports..

Alex

Hi

If you are looking for more low level functionality and a bit easier to experiment with, you may want to check out https://gitlab.com/voxl-public/utilities/voxl-rtsp/-/tree/dev . This tool is good for testing or running simple scenarios. This tool accepts some commands (like exposure control) via command line, so you could add a simple feature to save an image upon a command line input. Please see README to see how the exposure control input works and code for how it's implemented.

Please make sure that voxl-camera-server is disabled when you run voxl-rtsp.

If you still would like to go through the voxl-camera-server approach, we may need some more input from respective devs :).

I hope this helps..

Alex

@John-Nomikos-0 ,

In order to resolve your issue, I highly recommend to switch the ESC params to use our new smooth spin-up procedure. You can see how to enable it in the following esc param file (note that your motor control params are different, so don't just take this whole parameter file and use it):

https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-params/Sentinel_V1/Sentinel_V1.xml?ref_type=heads#L69

You will need to enter the kV value of the motor (motor_kv) and i would recommend updating the spinup_rpm_target to the rpm that motor would normally spin at like 10% power (usually in the range of 1000-3000 rpm). You can shorten the spinup time (spinup_time_ms) as 1500ms (1.5 seconds) was chosen in this case for a relatively large propeller with a lot of inertia. You can use 500ms as a good starting point for small motors/props. Having long spinup time will not hurt anything though.

If you see that during the smooth spinup, the motor seems to "slip" and stall, then increase the spinup_power slightly, by 10 units or so. This "slip" can happen if the spin-up power is very low and there is just not enough force to keep the motor spinning during the initial phase.

Also, please update min_num_cross_for_closed_loop to 5 as in this example. protection_stall_check_rpm should be set to less than minimum nominal RPM. So If the lowest normal RPM that the motor + propeller can spin (say at 7-10% power) is 2000rpm, I would set the stall check to 1000 or 1250 to prevent false triggers. This stall detection will protect the motor from burning out if it gets physically stuck due to some external event (hits something or gets tangled).

Please try this out and let us know if it resolves your problem!

Alex

@Aaky ,

You have good questions. Let me try to answer

1. We are working on some helper tools for our customer use to aid with evaluating the quality of focus. The tool will display an augmented image in voxl-portal, showing the results of the analysis (how sharp the image is). In general, you want to take a high contrast object (checkerboard), place it in the area where you want to focus and adjust the focus until the image is sharpest. The depth of field of a fisheye camera should be very large (everything beyond a small distance (a few inches) should be in focus). This means that placing an object 1-2 feet away from the camera can be easily used to calibrate the focus. I hope to share an initial version of the tool by the end of this week. Meanwhile, you may try to focus the camera by eye while having the camera pointed at a high contrast object

2. This "shadow" artifact happens for two reasons : fisheye lens blocks more light at extreme angles and also camera sensor CRA (chief ray angle) contributes to reduced sensitivity at large angles. The standard way to fix this in camera pipeline is using a lens-specific tuning file, which contains a Lens Shading Correction table and the ISP can do it without any CPU overhead. However, for tracking cameras, we use RAW images (not going through ISP image processing) to avoid any image processing artifacts, so this lens shading correction feature is not available. Nevertheless, the image can be corrected on the CPU and it should not be too expensive to do that. The best place to do that would be during the 10->8 bit conversion that happens on the CPU (camera sends 10 bit data and we convert to 8 bit to use for feature tracking). Using 10 bit data will provide better quality image and will save cpu cycles because the pixels will already be in memory. The 10->8 bit conversion is happening in the voxl-camera-server application. This feature is something we discussed internally, but we have not implemented it yet. the corners of a fisheye camera are usually of a lesser value because the features there are going to be stretched, lower quality (due to optics) and may not appear in the frame for a long time. Nevertheless, with corrected image brightness in the corners, the performance of VIO could be even better.

3. The exposure control algorithm that we implemented attempts to achieve a certain mean pixel (sample) value (MSV). it takes average of all pixels and if it is below the desired one, it increases gain and exposure to achieve the higher MSV. Gain and exposure both contribute to increasing image brightness, but they have their own side effects: long exposure will cause blur during motion and high gain will increase pixel noise. the QVIO algorithm applies a gaussian blur before doing feature detection / tracking, so it would be preferred to use higher gain than higher exposure and this is what we tuned for ov7251. I think going with those settings is a good start. However, the previous issue of dark corners can throw off the exposure control, driving image in the center to be too bright. The image you provided seems a bit too dark to my eye, perhaps you can increase the target MSV and see if that helps? were you able to track good features with QVIO using AR0144?

4. We are slowly switching over to using AR0144 cameras for tracking (VIO). In general M0149 camera module is a better camera module all around compared to ov7251 - better and higher resolution image sensor, better lens, so we expect better QVIO performance. I will double check with the team about any specific tuning for better QVIO performance using AR0144.

I am not sure how to resolve the authentication issue, but you can always use docker save myimage:latest | gzip > myimage_latest.tar.gz to a tar file, then docker load on another machine and push to docker hub.

Also, by the way, you can build your docker image for VOXL on a regular linux machine. Qemu allows you to emulate arm and arm64 architectures and Docker supports this. This is how the voxl-emulator runs on x86-64: So, basically you can build your docker image on a x86-64 host and then push it to docker hub or copy directly to VOXL.

I know that does not answer your question, but hopefully there is a workaround here..

Alex

.. or if you wanted to continue using voxl-camera-server for 4K, just modify the server to save yuv frames on-demand and convert to jpeg later, as we discussed above..

@John-Nomikos-0 , yes i will provide the default params that are loaded during production.

@MattK , can you please do a quick check in your /etc/modalai/voxl-camera-server.conf and see if any of your stereo pairs are set to be rotated in software? Specifically, look for:

			"en_rotate":	true,
			"en_rotate_second":	true,

I believe this is your issue.

if the rotate flag is set, this will cause the camera server to send a special command to the cameras to rotate the images (in the camera). code here. The cameras don't actually rotate the image, but the "readout direction" is reversed, so image is sent out instead of starting at top left, bottom right instead.

if you use combo-flip sensormodules and set en_rotate to false in camera server config, this will achieve exactly the same effect as non-flip sensormodules and en_rotate set to true.

Please note that if you rotate the image in the camera this way, you need to re-do the camera intrinsic calibration because the rotation will shift the principal points.

Once you undo the rotation on your front stereo, you should not have to re-do your extrinsics computations (w.r.t body) if you already did it assuming the cameras are right side up (you will simply be making the image consistent with your prior assumption of being right side up).

Alex

You can try docker save myimage:latest | gzip > <path_to_sd_card>/myimage_latest.tar.gz . <path_to_sd_card> should be /mnt/sdcard.

Note that SD card will be quite a bit slower than built-in EMMC, you might want to do some testing to find a fast and reliable SD card.

From personal experience the following SD cards have worked well and fast:

• Lexar Professional 1066x
• Kingston 64GB microSDXC Canvas Go Plus

However there are plenty of SD cards to choose from, you may want to run some SD card tests before you use it for something important. There are some fake SD cards out there which result in corrupt data.

Also, if you wanted Docker to use SD card for storing all images, check this article : https://docs.modalai.com/docker-on-voxl/#move-docker-image-from-data-to-mntsdcard

I double checked with other devs on the team and the jpeg snapshot feature in voxl-camera-server is still in development. I don't have a good ETA for the feature.

However, perhaps we can go back to evaluating voxl-rtsp for your application. If you disable hires camera in voxl-camera-server, you should be able to use voxl-rtsp with hires camera after voxl-camera-server has started for other cameras.

voxl-rtsp supports outputting jpegs. You can first confirm that you can access hires camera jpegs with voxl-rtsp (I don't think it can do 4K30 jpegs, but lower resolution/frame rate should work). That should work and then you could add an input to trigger voxl-rtsp to actually save a jpeg upon request (instead of every frame). This can be done similarly how exposure/gain commands are accepted by voxl-rtsp via a named pipe.

Hopefully this can work for you..