@Eric-Katzfey Okay, thanks again for all your help!

@Eric-Katzfey

Okay, then I'm gonna do that. Thanks for the assist.

@Zachary-Lowell-0 Thank you for your help and guidance. Is there a way to get the status information like battery voltage using Python?

@Eric-Katzfey Sure!

@AidanGallagher , I have updated the IMX412 drivers to support stable 4040x3040 60FPS.

Just for full disclosure, the camera in this mode is slightly overclocked from 2.1Gbps / lane to 2.2Gbps MIPI bit rate (voxl2 supports up to 2.5Gbps) in order to allow sufficient time between EOF and SOF. This is within 5% of the camera spec and seems to work fine.. If you do not feel comfortable using this mode, don't :). (only the specific 4040x3040 60fps mode has been changed).

We are going to continue testing this mode because it allows doing things like EIS at 4K60 without losing any vertical stabilization margin. If there are any issues with it, I will report back.

By the way, for best image quality, even if you don't need 4K output, but lets say 1080p, in the case of unstabilized video (no EIS), you should select 3840x2160 input and 1920x1080 output. The debayering process loses sharpness, so going from 1920x1080 RAW -> 1920x1080 RGB is going to be worse quality than 3840x2160 RAW -> 1920x1080 RGB. For unstabilized video you do not need the full frame because the top and bottom will be cropped out anyway, so reduce the waste and not select the full frame resolution (4040x3040). You can experiment using the different input resolutions to see what output quality (sharpness) you get. There is currently no option to do output sharpening when using MISP, but we will look into in the future. The down side of using higher resolution input is increased GPU usage (should not be an issue) and longer readout time (the rolling shutter artifacts will be more pronounced).

https://storage.googleapis.com/modalai_public/temp/imx412_test_bins/20250409/imx412_fps_eis_20250409_drivers.zip

- 4056x3040 @ 30 16.5ms readout time - 4040x3040 @ 30, 60 16.5ms readout time - 3840x2160 @ 30, 60, 80 11.8ms readout time - 1996x1520 @ 30, 60, 120 5.5ms readout time - 1936x1080 @ 30, 60, 90, 120, 240 4.0ms readout time - 1996x480 @ 30, 480 1.8ms readout time - 1996x240 @ 30, 800 0.9 - 1.0 ms readout time

@GlennTee Great!

@paul-ammer , sorry for the delay. Initial version of the document that addresses latency in wifi streaming has been published : https://docs.modalai.com/camera-video/low-latency-video-streaming/

There is a lot of information there, please take a look and let me know where you get stuck.

I suggest starting to experiment without updating the camera drivers (as discussed in the document), so you reduce number of variables at the beginning. You can also use the Qualcomm ISP pipeline (using hires_front_small_encoded stream) and we can discuss what MISP can be used for at another time.

In general, you will find that the majority of the delay is going to be on the receiver side, with some additional delay possible in the server side after the camera pipeline (buffering of the encoded frames before sending them out - I believe gstreamer rtsp server adds some delay. voxl-streamer is based on gstreamer).

Regarding the earlier question of unused streams: yes it is true that if streams like large_video, etc, are enabled but not used, they will not consume significant resources, however, if enabled, the stream specifications (dimensions) will play a role when the camera pipeline selects the camera operating mode, which should satisfy requirements of all streams. The most extreme case is when you enable the snapshot at highest possible resolution (but use the snapshot very rarely), and your video streaming resolution is relatively low - the camera pipeline will have to select the camera mode that provides the largest image size (which will be the slowest in terms of latency) and then will downscale to get the video stream of the desired (small) resolution.

Please let me know what other questions you may have. I will continue updating the document in order to clarify things and based on feedback. If other users have questions, feel free to follow up in this thread.

Alex

@Jskim
Finally, I tried a test.
I tried changing the video player and the Streamer died immediately.
The conclusion is that the Streamer died even when it was in the foreground.
플레이어 변경하면 바로 죽음.jpg

Thanks,

JS, Kim

@Jetson-Nano Yes, you can. You just need to make sure one is a "master" and one is a "slave"

@Alex-Kushleyev Thank you for the reply

Currently we are buying Voxl2 flight deck from you, I will be integrating in my drone after integrating what I need to do how to make it fly autonomously can you guide that how sdk works and how 3d map will be created how we need to plan a machine in the sdk

@tom It's okay, I managed to figure it out. Just had to change the RC protocol in the px4-conf file.

@Vinny
yes I am.

@LR The 5G configuration on Starling 2 is a fairly new offering so there may not be fully representative docs live.

@Mark-Ogata ,

Starling 2 Max already has 4 or 5 cameras (depending on configuration - 5th could be a TOF sensor if selected). There are two high resolution IMX412 cameras (up to 4056x3040 resolution rolling shutter RGB cameras) and there are two AR0144 monochrome global shutter cameras that are used for feature tracking / VIO (1280x800 resolution).

If you really wanted to, you could add two more high resolution cameras to VOXL camera port J8 (camera ports J6 and J7 are fully used up). Adding the first IMX412 camera would just require you to buy a camera with the correct adapter, such as M0161 camera (with ucoax cable) and M0155 adapter which would plug in to VOXL2's J8. In order to get two IMX412 cameras added to starling 2 Max, it will be a bit more complicated, including a small Kernel change.

It looks like you are potentially looking to re-configure cameras on Starling 2 Max, so my suggestion is first to understand the capabilities of the cameras you have (resolution, field of view, etc) . The lenses on the IMX412 cameras have about 120 degress of horizontal FOV (and you can check it) : https://www.modalai.com/products/msu-m0161 . So for full 360 degree coverage, you would technically only need three of these cameras. However, you have a total of 4 cameras already and the lower resolution AR0144 cameras also have a wide FOV (almost 140 degrees horizontal : https://docs.modalai.com/M0166/) and can be used to supplement the two high resolutions cameras.

It seems to me that you could take the cameras that you already have and build a hardware mount for them. Also, as Eric said you might want to consider using VOXL2 for whatever you are trying to do on the Raspberry pi, since VOXL2 runs Ubuntu linux (but no display).

In terms of USB adapters, your startling 2 max already has a wifi module that uses a USB2 adapter, so you can plug in an ethernet adapter instead of that wifi dongle. BTW M0151 adapter board (which is part of Starling 2 Max, this is were the wifi dongle is plugged in) supports USB 3.0 : https://www.modalai.com/products/m0151

Alex

@groupo Thanks for this! I am pretty familiar with the AT Commands I just haven't tried disabling bands yet. Will add it to me to-do list

@rdjarvis Okay, great! Instead of starting voxl-px4 with systemctl start voxl-px4 can you start it as a foreground process using voxl-px4 -d and then capture the output and paste as a response? I'd like to see if there are any error messages printed that could help determine why it isn't seeing your GPS unit.

@Alex-Kushleyev I agree. When I put in the USB3/UART Add on board, I see that there is some clearance b/w the JST connector and the LTE board.
So it looks like the LTE modem is seated in properly. Looking forward to a solution to my problem of LTE setup 🙂

@Nathan-Raras It seems that the libxml2 is up-to-date

Screenshot from 2025-04-16 13-24-16.png

@Alex-Kushleyev Thank you for reverting back.
I am using custom Board for the Lepton, with MCBL-00105-02 cable for the connection. I will verify the pinouts.
I checked the camera with different board and it is working now.

@zauberflote1 Hi,

So, it’s almost impossible for OpenVINS’ VIO to successfully reset while the drone is in flight if even a simple hand movement (which I always tried to do gently) is enough to prevent proper reinitialization, right?

What I used to do to break the VIO was cover the camera, wait for x, y, and z to return to zero, and then uncover it.

As for the test environment, it's rich in features, so I don't think that's the issue.

I increased the publish and tracking frequency because I wanted to ensure the VIO was "responsive," and with two or three cameras, I consistently achieved reliable results.