@SKA What pipes do you see listed in /run/mpa? If you run the command voxl-inspect-vio does it show that VIO is running okay?

@tom It's okay, I managed to figure it out. Just had to change the RC protocol in the px4-conf file.

@Eric-Katzfey Awesome, thanks for the feedback!

@GlennTee Great!

Hi @Tamilselvan , you can look at our Starling 2 drone, which comes with either 2 or 3 tracking cameras that can be used for VIO : https://www.modalai.com/products/starling-2 . This drone is designed for indoor navigation but also flies well outdoors. For longer flight times you might want to consider looking at Starling 2 Max.

Video using triple camera VIO on startling 2 : https://www.youtube.com/watch?v=12DWDV7XCiE

You could build your own drone using components similar to Starling 2, if you wanted. The main camera documentation for Starling Front End board (M0173) is here : https://docs.modalai.com/M0173/ which supports several camera options. M0173 plugs into VOXL2 directly and enables those camera configurations.

If you have a custom build drone, you would need to set up PX4 tuning and VIO (Open VINS) to work with your particular camera geometry. For standard drones we have default configuration files that are provided for convenience (so it will be more work to set up the SW after you build a custom drone).

Alex

@Mark-Ogata ,

Starling 2 Max already has 4 or 5 cameras (depending on configuration - 5th could be a TOF sensor if selected). There are two high resolution IMX412 cameras (up to 4056x3040 resolution rolling shutter RGB cameras) and there are two AR0144 monochrome global shutter cameras that are used for feature tracking / VIO (1280x800 resolution).

If you really wanted to, you could add two more high resolution cameras to VOXL camera port J8 (camera ports J6 and J7 are fully used up). Adding the first IMX412 camera would just require you to buy a camera with the correct adapter, such as M0161 camera (with ucoax cable) and M0155 adapter which would plug in to VOXL2's J8. In order to get two IMX412 cameras added to starling 2 Max, it will be a bit more complicated, including a small Kernel change.

It looks like you are potentially looking to re-configure cameras on Starling 2 Max, so my suggestion is first to understand the capabilities of the cameras you have (resolution, field of view, etc) . The lenses on the IMX412 cameras have about 120 degress of horizontal FOV (and you can check it) : https://www.modalai.com/products/msu-m0161 . So for full 360 degree coverage, you would technically only need three of these cameras. However, you have a total of 4 cameras already and the lower resolution AR0144 cameras also have a wide FOV (almost 140 degrees horizontal : https://docs.modalai.com/M0166/) and can be used to supplement the two high resolutions cameras.

It seems to me that you could take the cameras that you already have and build a hardware mount for them. Also, as Eric said you might want to consider using VOXL2 for whatever you are trying to do on the Raspberry pi, since VOXL2 runs Ubuntu linux (but no display).

In terms of USB adapters, your startling 2 max already has a wifi module that uses a USB2 adapter, so you can plug in an ethernet adapter instead of that wifi dongle. BTW M0151 adapter board (which is part of Starling 2 Max, this is were the wifi dongle is plugged in) supports USB 3.0 : https://www.modalai.com/products/m0151

Alex

Hi @Tamilselvan,

Yes, our FPV ESC should be a good match for this motor in terms of power, however you will need to perform tuning and testing (non-flying) to prevent de-syncs, as described here:

https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-tools/doc/low_kv_motor_tuning.md

You would first need to follow our standard tuning guide as described here, but first increase timing advance and sense advance to the numbers from the above link, so that you don't have a de-sync during calibration : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc/-/blob/master/voxl-esc-tools/calibration.md

Please note that the FPV ESC only has the total current sensing, so you would want to run the tests with the ESC ID2, which measures the total current, so it will be reported to the python test scripts.

Just for full disclosure, we do not do a lot of testing using low kV motors, but the parameters are there to adjust and test performance. You would need to do sufficient bench testing to convince yourself that the ESC tuning is good and you are ready to test flying. If you need more guidance for testing the ESC performance, I can help you. You should first start with the link I mentioned above and then modify the script to run continuous tests and perhaps detect anomalies. We may extend our test script to support testing like this with automatic error detection in the future.

If you do not have experience with ESC tuning or do not wish to go through this procedure, then our ESC may not be the best choice for this motor.

Alex

@SKA , this is most likely due to the max disparity value being too large - for 1280x800 resolution the maximum value is 32.

the max disparity parameter limit is as follows:

width <= 640 : max disparity = 64 (or less, in steps of 4) 640 < width <= 1280 : max disparity 32 (or less in steps of 4) 1280 < width <= 1920: 16, 12,8, or 4

So in your case, you should update your max disparity parameter to 32 or less.

I have been testing AR0144 DFS recently and i also found that i needed to increase the min_threshold and cost_threshold from the default of 10 that was used for OV7251. I found that values in the range of 50-80 produced good results.

Here is some additional information regarding DFS:

min_threshold:
Lower bound for adjusting occlusion cost threshold.
When set to < cost_threshold, the occlusion penalty
is varied linearly from this minimum value from the first row
to the last row.
The last row takes cost_threshold.
Range: 0 to 100
Recommended range: 50 to 75
min_threshold <= cost_threshold

cost_threshold:
Occlusion cost threshold.
The higher the value, more smoothness is enforced.
Range: 0 to 100
Recommended range: 50 to 75

Alex

@Eric-Katzfey Or update to a newer compiler. I just tried this one: https://developer.arm.com/-/media/Files/downloads/gnu/14.2.rel1/binrel/arm-gnu-toolchain-14.2.rel1-x86_64-aarch64-none-linux-gnu.tar.xz and it fixes that issue. But then it fails on a couple of other issue in septentrio.cpp that are easy to fix. So the solution is probably to upgrade the compiler and then fix the new reported errors in septentrio.cpp.

@Eric-Katzfey we are using A doodle radio

Just found that reboot -f works.

It's interesting weird that the network interface seems to not go down but all the services do - and don't come back.

UPDATE

I've simplified my snippet and I'm now able to "connect" to the VOXL2 via MAVSDK. It's still now working as I would expect it to, but first, here's my new script:

#!/usr/bin/env python3 import asyncio from mavsdk import System target_ip="192.168.1.187" target_port=14550 async def connect_to_drone(): print("Connecting to VOXL2...") # Create a System object with specific system ID and component ID # These match the IDs we're using in the heartbeat sender print("Creating System object...") drone = System() # Different connection string formats to try print("Connecting to drone...") await drone.connect(system_address=f"udp://{target_ip}:{target_port}") print("Waiting for drone to connect...") timeout = 60 # Extended timeout start_time = asyncio.get_event_loop().time() try: async for state in drone.core.connection_state(): print(f"Connection state: {state}") if state.is_connected: print(f"Connected to drone!") return drone # Check if we've timed out current_time = asyncio.get_event_loop().time() if current_time - start_time > timeout: print(f"Connection timed out after {timeout} seconds") break await asyncio.sleep(1) except Exception as e: print(f"Connection error: {e}") return None async def main(): # Try to connect to the drone drone = await connect_to_drone() # If connected, do something with the drone if drone and drone.core: try: # Print basic information info = await drone.info.get_version() print(f"Version Info: {info}") except Exception as e: print(f"Could not get info: {e}") if __name__ == "__main__": # Run the asyncio loop asyncio.run(main())

When I run this script I see only 3 logs:

Connecting to VOXL2... Creating System object... Connecting to drone...

When I adb shell and run voxl-inspect-mavlink mavlink_from_gcs I now see that the VOXL2 is receiving a heartbeat:

ID Mavlink MSG Name Counter Hz 0 heartbeat 5 1.1

But like I mentioned above, the code doesn't execute past await drone.connect(system_address=f"udp://{target_ip}:{target_port}").

I've tried udpout and nothing changes.

Any solutions? I'm having the same exact problem.

@groupo Thanks for this! I am pretty familiar with the AT Commands I just haven't tried disabling bands yet. Will add it to me to-do list

@tom That did the trick, thanks for the help!

Ok, thanks!

Something is in the works

We have managed to solve the transformations and now get the correct tag detection.
But even with fixed_frame_filter_len set to 1, QVIO and OpenVins are used and we see no relocalisation happening.

Does someone have a working example which we can look at, please?

@paul-ammer , sorry for the delay. Initial version of the document that addresses latency in wifi streaming has been published : https://docs.modalai.com/camera-video/low-latency-video-streaming/

There is a lot of information there, please take a look and let me know where you get stuck.

I suggest starting to experiment without updating the camera drivers (as discussed in the document), so you reduce number of variables at the beginning. You can also use the Qualcomm ISP pipeline (using hires_front_small_encoded stream) and we can discuss what MISP can be used for at another time.

In general, you will find that the majority of the delay is going to be on the receiver side, with some additional delay possible in the server side after the camera pipeline (buffering of the encoded frames before sending them out - I believe gstreamer rtsp server adds some delay. voxl-streamer is based on gstreamer).

Regarding the earlier question of unused streams: yes it is true that if streams like large_video, etc, are enabled but not used, they will not consume significant resources, however, if enabled, the stream specifications (dimensions) will play a role when the camera pipeline selects the camera operating mode, which should satisfy requirements of all streams. The most extreme case is when you enable the snapshot at highest possible resolution (but use the snapshot very rarely), and your video streaming resolution is relatively low - the camera pipeline will have to select the camera mode that provides the largest image size (which will be the slowest in terms of latency) and then will downscale to get the video stream of the desired (small) resolution.

Please let me know what other questions you may have. I will continue updating the document in order to clarify things and based on feedback. If other users have questions, feel free to follow up in this thread.

Alex