Posts made by Alex Kushleyev

@Matt69 , yes you should splice the cable as needed. I will double check if we have plans to release a new cable, it will probably depend on the demand.

Alex

@john_t , sounds good. The DFS processing on VOXL2 happens on the DSP, so it's quite efficient and does not use up much CPU or any GPU.

By the way, you could also simulate lower resolution DFS by binning the 1280x800 into 640x400 and it would be very similar to using the older M0015 stereo modules, so that DFS configuration from M0015 stereo could be used without much change.

Since the AR0144 stereo is not a configuration we currently use / support (but we have tested before), we don't have a ready-to-go documentation for setting this up.

This would be kind of experimental test on your end, which we will support. At minimum, you would need:

• voxl2 + APM + power adapter
• M0173
• two AR0144 (M0166) with appropriate coax cable lengths (comes with 80mm cable, but there are other lengths available via custom order)

If you wanted additional cameras, it may be chaper to get a M0173 bundle: such as https://www.modalai.com/products/m0173?variant=48528274424112 or similar (please check cable lengths)

Once you have the hardware, we can set up a similar test (based on your baseline, camera orientation) and help you get started.

Alex

@Jetson-Nano , can you please elaborate on what happened during the attempt to unbrick the voxl2 board?

@Matt69 , good question. let me ask someone else to help with that

Hello @newdroneflyer ,

In order to adjust focus of the hires camera, you need to do the following:

1. loosen the screw that holds the lens and prevents it from turning. this will allow the lens to turn within the assembly
2. turn the lens using the lens focus tool (should have come with your Starling 2), it is a black plastic tool that goes around the rim of the lens and allows you to turn the lens.
3. after focusing, tighten the screw that was loosened in step 1.

Here is the picture of the focus tool, it has two sizes - one for hires camera, one for tracking camera (if needed). You should have received one with the Starling2 (in an envelope, probably different packaging).

Alex

@Peter-Lingås-0

• some approximate perf specs for EIS are mentioned here : https://docs.modalai.com/camera-video/electronic-image-stabilization/#performance
  • in short, 4K60 (60fps) will use about 30% of the GPU, so two 4K30 would be similar GPU usage. CPU usage for EIS is minimal, as all the heavy lifting is done on the GPU.

Does your YOLOv8 run on CPU or GPU?

There is no documentation on binning, but it is actually very simple : when we talk about 2x2 binning, we typically mean (unless otherwise noted) that the binning happens on the camera itself. Here is a diagram that shows how the pixels are typically binned in the RGGB bayer pattern : https://www.1stvision.com/cameras/IDS/IDS-manuals/uEye_Manual/hw_binning.html

Because the camera does binning internally, this particular camera is able to do processing faster and there is less data to send out, so the readout time is reduced. Please note that not all cameras that output 2x2 binned images actually reduce the read time - some cameras are limited by the analog ADC part, which still has to sample all pixels and then bin them, so for those cameras you would get 2x2 binned image with the same readout time. The readout time is mainly driven by how fast the camera can read the pixels (ADC conversion), process them and send them out via MIPI. The MIPI output is usually not the bottleneck, but the analog/digital processing is.

There is no overhead for VOXL2 to received 2x2 binned image, in fact the power and cpu usage will be slightly reduced (less data to handle). However, you will lose detail if you use 2x2 binned image.

For analyzing small april tags in the distance, well, the smaller they are, the lower the rolling shutter skew will be (locally) so motion blur will probably dominate the distortion, so you would need to set the exposure low. The IMX412 has a good low-light sensitivity, so this allows you to reduce exposure and still get good image.

One last note, when using EIS, we typically use a full frame image (4056x3040, not 3840x2160), which is a 4:3 aspect ratio and provides a lot of margin for stabilization for 3840x2160 output. The readout time for the full frame image is proportionally larger (16ms instead of 12ms), however, locally the read out skew does not change (larger image = larger total rolling shutter skew). Also, EIS supports arbitrary output dimensions, so you can choose how much stabilization margin you want to have vs the stabilized FOV.

Out of curiosity, are you building a custom drone to support this effort or using one of our drones?

Alex

@Sarika-Sharma , thanks for the sending the picture of the TOF module.

In terms of connecting all 4 cameras to VOXL2 Mini, there is a lot of flexibility:

• both M0084 and M0135 provide two camera ports each
• any camera can be connected to any port, but i would probably group two tracking cameras together and then TOF + IMX214
• you could choose where to connect the cameras based on the mechanical integration requirements
• you can replace M0084 with another M0135 if you do not need a longer extension that M0084 provides
• depending on the actual locations of each camera, each camera will have a corresponding camera slot ID (0 through 3) and the camera drivers (sensormodules) will need to be set up correctly, but that is not difficult (we have a helper tools to help with that).
• a few notes about the TOF sensor
  • it needs external power (not from voxl), so you have to connect the red/black wire that branches off from the APM cable to the dark red power connector on the TOF module assembly
  • TOF module is pretty powerful and can potentially overheat if uncooled (with a heat spreader). Please check notes here:
  • https://docs.modalai.com/M0169/
  • https://docs.modalai.com/M0178/ (this is slightly newer version of the assembly, but should have some applicable info as well)
  • to prevent overheating, you can limit maximum "exposure" time for the ToF sensor (that is time the IR emitter is on), please see notes here : https://docs.modalai.com/M0178/#currentpower-consumption (the limit can be set in voxl-camera-server.conf.
  • TOF temperature is currently not possible to display anywhere, but if the TOF sensor overheats, the camera server will show an error and the sensor will stop working until until camera server is reset.
  • my suggestion is to limit the exposure to 2ms (2000us) instead of maximum 6ms in default mode if you do not have a heat spreader attached.
• you can see the heat spreader we now provide with the bundle here (https://www.modalai.com/products/m0178), screenshot below - heat spreader shown in lower left. (note this is the updated adapter configuration for tof sensor, not requiring external power, but we can work with what you already have).
  

Please make sure as you are plugging in all the interposers / flex cables (M0135 and M0084) and cameras that you line up pin1 locations (marked with a dot), so that you can make the connections properly.

Once you connect everything, you can send a picture of the board, flexes and cameras, i can double check the orientation and provide instructions how to actually set up the camera server for your config.

Alex

@jakkkkobo , we don't officially sell this part separately because we usually bundle it with M0173 on drone builds, however we can help you - please send us an email for a custom order.

Alex

Hello @Peter-Lingås-0 ,

I just checked and we have 0 stock of the color AR0144 cameras. At this moment, we do not have a plan to build more (due to lack of demand, as I mentioned). Unfortunately this means that we do not have any color global shutter cameras available.

Regarding rolling shutter, please keep in mind that in 2x2 binned mode, the 4.2ms skew is top to bottom across a pretty large vertical FOV (94 degrees for the lens used in IMX412 camera). Depending on the size of the object that you are looking for, you can calculate the distortion due to motion + rolling shutter skew.

EIS can correct for camera rotation, however motion compensation would require depth information and is a lot more complicated. We do support EIS on voxl2:

• https://docs.modalai.com/camera-video/electronic-image-stabilization/
• here is the performance you might expect (from tests some time ago):
  • dual camera EIS on Starling 2 Max (hand carried) : https://www.youtube.com/watch?v=-BA_nU4kjQs
  • down-facing camera EIS on Starling 2 Max (flight) : https://www.youtube.com/watch?v=-B5xKmBBCAc

You can see the rolling shutter effects present in the hand carried video. The down-facing video demo is a flight at about 5m, actually, might be close to what you are looking for. Both videos were actually shot in 4K (4056x3040 full frame) input mode (longer readout time) and then downscaled to 1080p during EIS (but EIS can also produce 4K output).

Additional notes:

• lower exposure time will not reduce rolling shutter skew, it will reduce motion blur
• 2x2 binned mode automatically gives you 1920x1080 instead of 3840x2160, however you can also get 3840x2160 (or even full frame) and re-scale the image down. You will get better image quality (because the source image is higher resolution), but will have higher rolling shutter skew (12ms vs 4ms -- note it's not exactly 4 times difference due to how the camera operates internally).

Since we don't have an option for global shutter color camera, and the EIS performance looks acceptable, you could try going with IMX412 + EIS (single or dual IMX412 cameras). BTW, VOXL2 can run dual EIS as well, as shown in one of the videos. EIS can work with either full resolution or 2x2 binned as input and output resolution can be arbitrary.

Alex

Hi @JoonaR,

Yes, the Starling 2 drones are shipped with the lepton adapter (M0187), but Lepton sensor itself is not included.

https://docs.modalai.com/M0187/

Alex

Hello @Sarika-Sharma ,

Do you have all these cameras already and are asking how to connect them, or are you looking to purchase them?

If you do have the M0169, can you please post the picture of the TOF module and attached adapters that came with it? I just want to make sure I know exactly what you have, since there are several adapter options.

Also, are you open to any updates to the camera selection, as I can make suggestions to use better cameras (both tracking cameras and hires cameras).

Alex

@Piyush-Singh, please double check that you have the imx214 sensormodule bin for slot 0 in /usr/lib/camera/ (since you have M0084 connected to J6 and the IMX214 camera connected to the "lower" port (JL) of M0084, so the resulting connection of the IMX214 camera will be mapped to VOXL2's J6L (slot 0)).

You can find all the sensormodule files in /usr/share/modalai/chi-cdk/

Alex

Hello @SKA,

Thank you for the details. I can't think of anything right now, but will double check this.

I know that there is one potential issue in voxl-camera-server - the SPS header (which is part of the H264 / H265 stream) is only sent by the video encoder once (at the beginning of the stream). I believe that voxl-streamer will cache the SPS packet (which contains things like resolution, etc) and i think when clients connect to voxl-streamer, then it will send that SPS header to the clients. However, if voxl-streamer does not get that first header, it will not be able to forward it to the clients. This is a guess for now, but could be related to the issue that you are seeing.

I can look into this issue. Can you please let me know how exactly to set up VLC for 0ms cache? I want to make sure that my test is the same. Also, which version of VLC are you using?

Also, i noticed one thing - you are using hires_misp_color stream from camera server, which is not actually encoded, but YUV. Then voxl-streamer will take those YUVs and encode them (using hardware encoder). Is there a reason why you are not using the hires_misp_encoded stream to get H264 directly from camera server and serve it using voxl-streamer without re-encoding? I wonder if in the old SDK you were using _color or _encoded stream?

Alex

@cbay ,

As long as the Boson / Boson+ 640 (or 320) supports MIPI output, it should work. We believe that some Boson models do NOT have MIPI support (HW limitation) but all Boson+ do have MIPI support. Please check with your vendor to make sure MIPI output is supported.

Also, did you actually mean M0187, or something else? M0187 is an adapter for a Flir Lepton : https://docs.modalai.com/M0187/

Alex

Hello @MattB69 ,

Sorry I made a mistake regarding the current rating of the step-up regulator. the 80mA number was for a very low input voltage (to the step-up regulator). with 3.3V intput to the step-up regulator, M0065 can support up to 350mA of output current at 5V, but we still do not recommend using it for powering servos.

So this means that our documentation for D0015 flight configuration is in conflict with documentation for M0065 board (which says use 5.0V for reference only).

Even though we have tested it in the D0015 configuration with "some" servos (i am not sure which ones were used), we still do not recommend powering servos using 5V from M0065. We will update the D0015 documentation.. Thanks for pointing it out!

Alex

Hi @Martin-Lukac ,

Thank you for the update. I am glad to see that your new tests went fine. As I mentioned before, i was quite surprised to see the ESC fail in such a way, which may suggest some kind of anomaly.

I will provide a longer answer soon, but for now some quick notes:

• non-zero kp / ki parameters will speed up the ESC's RPM response. I will provide some examples
• you can test whether the ESC closed-loop controller is stable using voxl-esc tools, using step inputs, etc. if unstable, lower Kp. Then lower Ki to avoid slow overshoots. Ki will definitely not contribute to high frequency oscillations.
• assuming the ESC closed loop response is stable, if the Flight Controller is commanding noisy RPMs, the ESC with aggressive tuning (high kp) will try to track the noisy rpm commands, resulting in motor and ESC heating up.
• there are generally two main sources of oscillations: unstable attitude controller or vibrations / noisy gyro
  • unstable attitude controller : characterized by lower frequency oscillations (in the range of 10-30Hz, depending on your frame, motors, etc)
  • vibrations can result from drone frame twisting at the frequency of motor RPM, or loose flight controller board / IMU, if flight board is shock mounted, the connected wires can potentially carry vibrations or cause the board to flex due to the wires vibrating / moving (very quickly). These vibrations will be at the frequency of motor RPM. aggressive (high) values of ESC kp will amplify these vibrations, so you can end up in a situation when the ESC's closed loop RPM controller is feeding back and contributing to higher oscillations.
  • by analyzing the gyro logs (will need to enable high rate logging), using FFT you can determine if your frame is mechanically unstable / noisy or the attitude controller is unstable.
• if you set kp and ki to 0, you still have the advantage of calibrated (and voltage-compensated) RPM control. However, the control will only have feed-forward term. So behavior will be close to response of a traditional ESC with added calibrated feed-forward rpm control with voltage compensation.
• regarding whether you need a bigger ESC.. it depends on the maximum total current that you plan to pull. The M0138 ESC has been tested with similar motor / prop combo (3110 motor, 900kV with a 10x4.5 tri-blade) . During bench testing, each motor could pull close to 40A at 6S, however in practice you are never going to have all 4 motors spinning at absolute maximum rpm (then you will not have any control margin for attitude). We have tested full punch-out flights for 100+ Amps for over a minute and the ESC was perfectly fine (airflow required for cooling). The mosfets on M0138 are rated for 100A+ continuous (each), so 40A continuous an issue. It will come down to the total current draw and cooling. For high power applications like this, you should definitely start at lighter loads and try to push the ESC to the limit. You will also need to worry about power connector(s) if you are drawing 100A+ continuously (battery connector may melt).
• px4 params: the min and max rpm params in px4 should match the min and max in your esc params. the px4 params do not override the limits in the esc params - the ESC will cap the incoming rpm commands if they are outside of the ESC params limits. Also, if you have incorrect px4 rpm limits, the thrust calculation in px4 will be incorrectly scaled from 0 to 100%, so you definitely want to get those rpm limits to be in sync between the ESC and px4.

Also, you can use our custom version of flight-review to display additional data specific to our ESC (rpm commands, actual rpms, esc temps, etc) : https://github.com/modalai/px4-flight-review (you can run it locally in docker).

Alex

Hi @SKA ,

Can you please clarify a few things:

• using latest SDK, does rtsp streaming work, if you do NOT specify 0ms cache?
• what player are you using for receiving the video? have you tried ffmpeg (if so, what is the full command?)
• are you sure that you are actually have the hires_misp_color stream available from the camera server? you can check it using either voxl-portal or using voxl-inspect-cam hires_misp_color.

Alex

@MattB69 ,

The 5V output on M0065 (10-pin connector with pwm outputs) should not be used to power any servos. The 5V signal is generated from 3.3V (coming from VOXL2) using a step-up converter, only capable of 80mA output).

https://docs.modalai.com/voxl2-io-datasheet/

the 3.3V (which is coming from VOXL2), also should not be used to power servos, as that would be risking bringing down a major power rail on VOXL2.

Alex

@Gary-Holmgren ,

The configuration you are asking about is the standard C28 (https://docs.modalai.com/voxl2-coax-camera-bundles/#mdk-m0173-1-03) with an addition of another hires camera.

Using MISP (not built-in ISP), VOXL2 supports as many hires cameras as can be physically connected (may run into some limitations if you are running at full resolution and encoding all streams at the same time).

C28 config uses VOXL2 + M0173 and you are left with voxl2 J8 (which has two camera ports), so we just need to add another IMX412 to voxl2 J8. The best way to do this would be using M0181 (which supports a Boson + hires camera), it would be connected like this (you can just ignore Boson):

What is not shown here is the standard C28 config (using M0173 adapter, plugged into VOXL2 J6, J7).

Just to make sure there will be enough processing resources for the three IMX412, can you briefly describe what resolution you plan to use and whether you need encoded video, etc. I could test this configuration for you.

Alex

@YUUJI-INOUE ,

Yes the square glass piece is the IR cut filter. It should be glued to the lens assembly in the for corners of the square. You could remove it yourself, at your own risk. Please see a similar discussion and results in the following thread : https://forum.modalai.com/topic/4826/msu-m0149-1-ir-filter (this that case, the lens is slightly different and it was much more difficult to remove the IR filter). In your case, the filter may pop out without breaking, but it is likely to break. You may want to put some tape on the filter before removing it, which would help avoid having loose pieces of glass.

Alex