Posts made by ctitus

@Zachary-Lowell-0 Perfect, thanks! I'll try it when I get some time next week.

@groupo To clarify, my setup is as follows: I have 2 PX4 drones flying and connected to QGC (vehicle 201 and vehicle 202, numbers set by MAV_SYS_ID). They both have cameras and live streams set up at rtsp://xxx.xxx.xxx.201:8900/live and rtsp://xxx.xxx.xxx.202:8900/live respectively. I can use QGC to switch between vehicle 201 and 202's telemetry, but doing this doesn't switch the live stream between the drones. What I'd like to have happen is when I switch from vehicle 201 to 202, I'd like the live stream displayed on QGC to switch from vehicle 201 to 202 as well.

Maybe I have something configured wrong, but currently the way I've been getting around this is going into QGC Applications Settings > General and modifying the RTSP Stream URL to match whichever drone I'd like to view.

I found this old post here about implementing getting RTSP streaming fully integrated with QGC. It says it requires a MAV_SYS_ID = 1, but was also from 3 years ago and uses a much older QGC. We have multiple drones and would like to switch between livestreams. I've seen other systems with the ability to switch between multiple live streams (from different vehicles) based off of MAV_SYS_ID, so when I switch to vehicle 201 in QGC, my livestream will automatically switch to that vehicle's as well. Is this something that is currently supported or will be in the future? If not, do you have a recommended alternative? Worst case, I can just ffplay both of them, use two VLC windows, or manually switch RTSP addresses in QGC. It's not a big deal, but it would be nice to have multiagent support in a more integrated way.

I'm using VOXL2, SDK 1.3.3, system image 1.7.8, QGC 4.4.2*, host machine is Ubuntu 22.04 LTS
*I can't go to the older QGC because there's an issue with rally points and geofences with wildly different PX4/QGC versions and we need to be able to download and view the geofences we set flight safety reasons

@Alex-Kushleyev I was able to get it working with some changes to hal3_camera_mgr.cpp. The short version is I added int32_t aeExposureCompensation = -12; requestMetadata.update(ANDROID_CONTROL_AE_EXPOSURE_COMPENSATION, &aeExposureCompensation, 1); to the AE_ISP case here in hal3_camera_mgr.cpp and recompiled and deployed the voxl-camera-server following the docs here. I also had a hard time finding the docs for these parameters for some reason so I'll like them here just in case it's useful to others.

The slightly longer version is I really added a custom auto exposure mode AE_CUSTOM to lines 169-189 of common_defs.h, and then copied the AE_ISP case, renamed it custom, and added the two lines above to that. With this method, I also had to modify this line and add if(ae_mode != AE_ISP && ae_mode != AE_CUSTOM){ to prevent the exposure and gains from being overwritten. I'd like to add the autoexposure compensation as a parameter to /etc/modalai/voxl-camera-server.conf so it's more convenient to change, but it's not urgent.

If there's a better way to do any of this, I'd appreciate any feedback, but it is working now. I've attached my /etc/modalai/voxl-camera-server.conf below

 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[{
			"type":	"imx412",
			"name":	"hires",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"en_raw_preview":	false,
			"en_small_video":	true,
			"small_video_width":	1024,
			"small_video_height":	768,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	true,
			"large_video_width":	4056,
			"large_video_height":	3040,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	true,
			"en_snapshot_width":	4056,
			"en_snapshot_height":	3040,
			"exif_focal_length":	3.0999999046325684,
			"exif_focal_length_in_35mm_format":	17,
			"exif_fnumber":	1.2400000095367432,
			"ae_mode":	"custom"
		}]
}```

If there's a way to modify the very start of the pipe to set autoexposure=1 and ae-compensation=-12 and then pipe that back into VOXL's pipe architecture, or to maybe turn off any voxl sdk autoexposure and just modify qualcomm's built in parameters, that would work just fine for me, I would just need to know where to look to find where the qualcomm camera parameters are set. Even if it's not a pretty solution, a temporary hardcoded fix I can implement would be just fine for now.

@zauberflote1 I also tried changing the exposure with an old command, but there was no change

voxl-send-command hires_snapshot set_exp_gain 1 1 ; voxl-send-command hires_snapshot snapshot

The isp is too bright and the apriltag is more washed out than I would like, the lme_msv picture was with the ridiculously low exposure times and looks the same regardless of how I change the parameters.

On the same camera with a different drone, I've run the following command to get the image quality I'd like:

gst-launch-1.0 -e qtiqmmfsrc camera=0 autoexposure=1 ae-compensation=-12 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=1/2 ! videoflip method=vertical-flip ! queue ! jpegenc ! multifilesink location="/data/images/img.jpg" max-files=1

isp_aerial_apriltag.png vvv

isp.png vvv

lme_msv.png vvv

@zauberflote1 Thanks, I appreciate it!

voxl-streamer.conf

/**
 * This file contains configuration parameters for voxl-streamer.
 *
 * input-pipe:
 *    This is the MPA pipe to subscribe to. Ideally this is a pipe
 *    with an H264 stream such as the default: hires-stream
 *    However, you can point voxl-streamer to a RAW8 uncompressed stream such as
 *    tracking or qvio_overlay. In this case voxl-streamer will encode the stream
 *    at the bitrate provided in the birtate field.
 *    if input-pipe is already H264 then the bitrate config here is ignored and you
 *    should set the bitrate in voxl-camera-server.conf!!!!!
 *
 * bitrate:
 *    Bitrate to compress raw MPA streams to.
 *    Ignored for H264 streams like hires_stream
 *
 * decimator:
 *    Decimate frames to drop framerate of RAW streams.
 *    Ignored for H264 streams like hires_stream
 *
 * port:
 *    port to serve rtsp stream on, default is 8900
 *
 */
{
	"conf-version":	"1.4",
	"configuration":	"hires",
	"uvc-generic":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"uvc"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"decimator":	1,
				"bitrate":	1000000
			}
		}
	},
	"uvc-flir-boson":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"uvc"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	512,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"uvc-flir-lepton":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"uvc"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	160,
				"height":	120,
				"bitrate":	500000
			}
		}
	},
	"hires":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"hires"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"hires-logo":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"hires"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		},
		"overlay":	{
			"location":	"/etc/modalai/modalai.png",
			"offset_x":	-1,
			"offset_y":	-1
		}
	},
	"stereo":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"stereo"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	960,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"stereo-color":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"stereo_front"
		},
		"output":	{
			"stream":	{
				"width":	1280,
				"height":	800,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"tracking":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"tracking"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"dfs-disparity":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"dfs_disparity"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"qvio-overlay":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"qvio_overlay"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	544,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"tflite-overlay":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"tflite"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"hdmi-mpa":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"v4l2"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	360,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"gphoto2-mpa":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"gphoto2"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	360,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"video-test":	{
		"input":	{
			"interface":	"test",
			"frame":	{
				"width":	640,
				"height":	480,
				"format":	"yuv420"
			}
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"rate":	30,
				"bitrate":	1000000
			}
		}
	},
	"uvc-video":	{
		"input":	{
			"interface":	"uvc",
			"device":	"/dev/video0"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	360,
				"rate":	15,
				"bitrate":	1000000
			}
		}
	},
	"input-pipe":	"hires_small_encoded",
	"bitrate":	1000000,
	"rotation":	0,
	"decimator":	1,
	"port":	8900
}

voxl-camera-server.conf

/**
 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[{
			"type":	"imx412",
			"name":	"hires",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"en_raw_preview":	false,
			"en_small_video":	true,
			"small_video_width":	1024,
			"small_video_height":	768,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	true,
			"large_video_width":	4056,
			"large_video_height":	3040,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	true,
			"en_snapshot_width":	4056,
			"en_snapshot_height":	3040,
			"exif_focal_length":	3.0999999046325684,
			"exif_focal_length_in_35mm_format":	17,
			"exif_fnumber":	1.2400000095367432,
			"ae_mode":	"isp"
		}]
}

Also, related question, where can I do vertical and horizontal flipping of the image?

@zauberflote1 Thanks! I tried doing this and the voxl camera server and streamer services start up just fine, but it is so overexposed everything is basically white. It seems like my changes to the libmodal exposure settings have no effect on this which makes me think I'm missing something (I'm restarting the camera server and streamer after saving my conf file changes).

I would expect this to give extremely dark pictures, but it's just as white as the defaults

"ae_desired_msv":       10,
"exposure_min_us":      20,
"exposure_max_us":      30,
"gain_min":     0,
"gain_max":     30,
"exposure_soft_min_us": 0,
"ae_filter_alpha":      0.600000023841858,
"ae_ignore_fraction":   0.20000000298023224,
"ae_slope":     0.05000000074505806,
"ae_exposure_period":   1,
"ae_gain_period":       1

I'm using an IMX412 on VOXL2, and the pictures are a little too exposed. I'd like to set the exposure-compensation to -12 (https://docs.qualcomm.com/bundle/publicresource/80-50450-50_REV_AC_Qualcomm_Intelligent_Multimedia_SDK__QIM_SDK__Reference.pdf). What is the intended way to do this with the VOXL2 SDK 1.3.3 (sysimg 1.7.8)

/etc/modalai/voxl-camera-server.conf below:
{
"version": 0.1,
"fsync_en": false,
"fsync_gpio": 109,
"cameras": [{
"type": "imx412",
"name": "hires",
"enabled": true,
"camera_id": 0,
"fps": 30,
"en_preview": false,
"preview_width": 640,
"preview_height": 480,
"en_raw_preview": false,
"en_small_video": true,
"small_video_width": 1024,
"small_video_height": 768,
"small_venc_mode": "h265",
"small_venc_br_ctrl": "cqp",
"small_venc_Qfixed": 30,
"small_venc_Qmin": 15,
"small_venc_Qmax": 40,
"small_venc_nPframes": 9,
"small_venc_mbps": 2,
"en_large_video": true,
"large_video_width": 4056,
"large_video_height": 3040,
"large_venc_mode": "h265",
"large_venc_br_ctrl": "cqp",
"large_venc_Qfixed": 38,
"large_venc_Qmin": 15,
"large_venc_Qmax": 50,
"large_venc_nPframes": 29,
"large_venc_mbps": 30,
"en_snapshot": true,
"en_snapshot_width": 4056,
"en_snapshot_height": 3040,
"exif_focal_length": 3.0999999046325684,
"exif_focal_length_in_35mm_format": 17,
"exif_fnumber": 1.2400000095367432,
"ae_mode": "isp"
}]
}

@Vinny @tom Thanks for all the help. I have the doodle powered on a different bus, I just have the signal going to the J12. And modifying the voxl-configure-modem script worked for me, I now have custom IPs on eth1 like I wanted.

Hi,

I'm using the VOXL2 with the VOXL 2 Ethernet and USB Hub Add-on board, and I have wifi on the USB3 port (J11), a doodle radio on the USB2 port (J12), and keep the ethernet port open for hardwiring with an ethernet cable (J13). My problem is when I use the voxl-configure-modem command to set the static ip for the doodle, I cannot ping it. It works just fine using the USB3 UART Add-On, and I think this is because that board only exposes eth0. However, the Ethernet and USB Hub Add-on board has both eth0 and eth1. If I run ip addr add 10.223.1.101/16 dev eth1, I am able to ping the doodle and it works just fine.

My question is what is the intended method (if any) to essentially run voxl-configure-modem such that it configures everything for eth1 instead of eth0? If there is no way to do that, how would you recommend I persist a static ip on eth1 though reboots?

@tom it looks like the groups are the same between machines (and yours except I'm not in docker), and the versions of adb, fastboot, and all android packages are the same. I was looking online and some people say it may be related to or solved by udev (https://askubuntu.com/questions/705537/no-permissions-fastboot-fastboot-and-adb-not-working) but I'm not sure I have more more time to troubleshoot this, and I didn't touch anything with udev on the working machine. Hopefully I'll be able to circle back to this eventually, but thank's for your help with everything!

@tom Both are linux ubuntu 22.04 LTS. My current laptop I've been using for a while and likely have some weird issue in my user setup or configuration, the new laptop was dual booted as linux for the first time 2 hours ago and the first thing it did was factory reset the voxl and run install.sh (and install the missing dependencies that install.sh alerted me to) which worked just fine without needing sudo. I'm not sure what I could've done to my laptop to have this issue, but any advice would be greatly appreciated

And running sudo ./install.sh worked and successfully installed the SDK and system image

@ctitus Messing aroudn a bit more, sudo fastboot getvar "partition-size:modalai_conf" works

@tom So I did some digging and found where it hangs. Line 391 of voxl2_1.3.1-0.8/system-image/flash-system-image.sh: local output="$( fastboot getvar "partition-size:${partition}" 2>&1 )" specifically executing fastboot getvar "partition-size:modalai_conf". For some reason, my original laptop shows "< waiting for any device >" while the laptop that worked returned "partition-size:modalai_conf: 0x4000000". I double checked that the voxl shows up as fastboot with lsusb on both laptops

@tom No, it's only ever 1 at a time. I was able to get my hands on a new fresh linux machine, and after installing the packages I needed (and using the same files I copy and pasted from my original laptop), I was successfully able to factory reset and roll back the sdk and system image without issue. I'm gonna do a bit of digging but I have a feeling I was just missing some packages and it wasn't alerting me of it on my original laptop.

@ctitus I've been trying more sdk's and it seems like none of them will flash properly, but there's no error or anything thrown, just nothing happens.