@Zachary-Lowell-0 Perfect, thanks! I'll try it when I get some time next week.

@groupo To clarify, my setup is as follows: I have 2 PX4 drones flying and connected to QGC (vehicle 201 and vehicle 202, numbers set by MAV_SYS_ID). They both have cameras and live streams set up at rtsp://xxx.xxx.xxx.201:8900/live and rtsp://xxx.xxx.xxx.202:8900/live respectively. I can use QGC to switch between vehicle 201 and 202's telemetry, but doing this doesn't switch the live stream between the drones. What I'd like to have happen is when I switch from vehicle 201 to 202, I'd like the live stream displayed on QGC to switch from vehicle 201 to 202 as well.

Maybe I have something configured wrong, but currently the way I've been getting around this is going into QGC Applications Settings > General and modifying the RTSP Stream URL to match whichever drone I'd like to view.

I found this old post here about implementing getting RTSP streaming fully integrated with QGC. It says it requires a MAV_SYS_ID = 1, but was also from 3 years ago and uses a much older QGC. We have multiple drones and would like to switch between livestreams. I've seen other systems with the ability to switch between multiple live streams (from different vehicles) based off of MAV_SYS_ID, so when I switch to vehicle 201 in QGC, my livestream will automatically switch to that vehicle's as well. Is this something that is currently supported or will be in the future? If not, do you have a recommended alternative? Worst case, I can just ffplay both of them, use two VLC windows, or manually switch RTSP addresses in QGC. It's not a big deal, but it would be nice to have multiagent support in a more integrated way.

I'm using VOXL2, SDK 1.3.3, system image 1.7.8, QGC 4.4.2*, host machine is Ubuntu 22.04 LTS
*I can't go to the older QGC because there's an issue with rally points and geofences with wildly different PX4/QGC versions and we need to be able to download and view the geofences we set flight safety reasons

@Alex-Kushleyev I was able to get it working with some changes to hal3_camera_mgr.cpp. The short version is I added int32_t aeExposureCompensation = -12; requestMetadata.update(ANDROID_CONTROL_AE_EXPOSURE_COMPENSATION, &aeExposureCompensation, 1); to the AE_ISP case here in hal3_camera_mgr.cpp and recompiled and deployed the voxl-camera-server following the docs here. I also had a hard time finding the docs for these parameters for some reason so I'll like them here just in case it's useful to others.

The slightly longer version is I really added a custom auto exposure mode AE_CUSTOM to lines 169-189 of common_defs.h, and then copied the AE_ISP case, renamed it custom, and added the two lines above to that. With this method, I also had to modify this line and add if(ae_mode != AE_ISP && ae_mode != AE_CUSTOM){ to prevent the exposure and gains from being overwritten. I'd like to add the autoexposure compensation as a parameter to /etc/modalai/voxl-camera-server.conf so it's more convenient to change, but it's not urgent.

If there's a better way to do any of this, I'd appreciate any feedback, but it is working now. I've attached my /etc/modalai/voxl-camera-server.conf below

 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[{
			"type":	"imx412",
			"name":	"hires",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"en_raw_preview":	false,
			"en_small_video":	true,
			"small_video_width":	1024,
			"small_video_height":	768,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	true,
			"large_video_width":	4056,
			"large_video_height":	3040,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	true,
			"en_snapshot_width":	4056,
			"en_snapshot_height":	3040,
			"exif_focal_length":	3.0999999046325684,
			"exif_focal_length_in_35mm_format":	17,
			"exif_fnumber":	1.2400000095367432,
			"ae_mode":	"custom"
		}]
}```

If there's a way to modify the very start of the pipe to set autoexposure=1 and ae-compensation=-12 and then pipe that back into VOXL's pipe architecture, or to maybe turn off any voxl sdk autoexposure and just modify qualcomm's built in parameters, that would work just fine for me, I would just need to know where to look to find where the qualcomm camera parameters are set. Even if it's not a pretty solution, a temporary hardcoded fix I can implement would be just fine for now.

@zauberflote1 I also tried changing the exposure with an old command, but there was no change

voxl-send-command hires_snapshot set_exp_gain 1 1 ; voxl-send-command hires_snapshot snapshot

The isp is too bright and the apriltag is more washed out than I would like, the lme_msv picture was with the ridiculously low exposure times and looks the same regardless of how I change the parameters.

On the same camera with a different drone, I've run the following command to get the image quality I'd like:

gst-launch-1.0 -e qtiqmmfsrc camera=0 autoexposure=1 ae-compensation=-12 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=1/2 ! videoflip method=vertical-flip ! queue ! jpegenc ! multifilesink location="/data/images/img.jpg" max-files=1

isp_aerial_apriltag.png vvv

isp.png vvv

lme_msv.png vvv

@zauberflote1 Thanks, I appreciate it!

voxl-streamer.conf

/**
 * This file contains configuration parameters for voxl-streamer.
 *
 * input-pipe:
 *    This is the MPA pipe to subscribe to. Ideally this is a pipe
 *    with an H264 stream such as the default: hires-stream
 *    However, you can point voxl-streamer to a RAW8 uncompressed stream such as
 *    tracking or qvio_overlay. In this case voxl-streamer will encode the stream
 *    at the bitrate provided in the birtate field.
 *    if input-pipe is already H264 then the bitrate config here is ignored and you
 *    should set the bitrate in voxl-camera-server.conf!!!!!
 *
 * bitrate:
 *    Bitrate to compress raw MPA streams to.
 *    Ignored for H264 streams like hires_stream
 *
 * decimator:
 *    Decimate frames to drop framerate of RAW streams.
 *    Ignored for H264 streams like hires_stream
 *
 * port:
 *    port to serve rtsp stream on, default is 8900
 *
 */
{
	"conf-version":	"1.4",
	"configuration":	"hires",
	"uvc-generic":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"uvc"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"decimator":	1,
				"bitrate":	1000000
			}
		}
	},
	"uvc-flir-boson":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"uvc"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	512,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"uvc-flir-lepton":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"uvc"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	160,
				"height":	120,
				"bitrate":	500000
			}
		}
	},
	"hires":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"hires"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"hires-logo":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"hires"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		},
		"overlay":	{
			"location":	"/etc/modalai/modalai.png",
			"offset_x":	-1,
			"offset_y":	-1
		}
	},
	"stereo":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"stereo"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	960,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"stereo-color":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"stereo_front"
		},
		"output":	{
			"stream":	{
				"width":	1280,
				"height":	800,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"tracking":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"tracking"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"dfs-disparity":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"dfs_disparity"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"qvio-overlay":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"qvio_overlay"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	544,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"tflite-overlay":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"tflite"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	480,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"hdmi-mpa":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"v4l2"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	360,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"gphoto2-mpa":	{
		"input":	{
			"interface":	"mpa",
			"mpa-camera":	"gphoto2"
		},
		"output":	{
			"stream":	{
				"width":	640,
				"height":	360,
				"decimator":	2,
				"bitrate":	1000000
			}
		}
	},
	"video-test":	{
		"input":	{
			"interface":	"test",
			"frame":	{
				"width":	640,
				"height":	480,
				"format":	"yuv420"
			}
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	480,
				"rate":	30,
				"bitrate":	1000000
			}
		}
	},
	"uvc-video":	{
		"input":	{
			"interface":	"uvc",
			"device":	"/dev/video0"
		},
		"output":	{
			"stream":	{
				"rotation":	0,
				"width":	640,
				"height":	360,
				"rate":	15,
				"bitrate":	1000000
			}
		}
	},
	"input-pipe":	"hires_small_encoded",
	"bitrate":	1000000,
	"rotation":	0,
	"decimator":	1,
	"port":	8900
}

voxl-camera-server.conf

/**
 * voxl-camera-server Configuration File
 *
 * Each camera has configurations for up to 4 HAL3 streams:
 *    - `preview` stream for raw unprocessed images from CV cameras
 *    - `small_video` 720p (ish) h264/h265 compressed for fpv video streaming
 *    - `large_video` 4k (ish) h264/h265 for onboard video recording to disk
 *    - `snapshot` ISP-processed JPG snapshots that get saved to disk
 *
 * on QRB5165 platforms (VOXL2 and VOXL2 mini) you can only have 3 of the 4 enabled
 *
 * This file is generated from default values by voxl-configure-cameras.
 * Do not expect arbitrary resolutions to work, the ISP and video compression
 * pipelines only support very specific resolutions.
 *
 * The default video compression mode is cqp or Constant Quantization Parameter
 *
 *
 *
 */
{
	"version":	0.1,
	"fsync_en":	false,
	"fsync_gpio":	109,
	"cameras":	[{
			"type":	"imx412",
			"name":	"hires",
			"enabled":	true,
			"camera_id":	0,
			"fps":	30,
			"en_preview":	false,
			"preview_width":	640,
			"preview_height":	480,
			"en_raw_preview":	false,
			"en_small_video":	true,
			"small_video_width":	1024,
			"small_video_height":	768,
			"small_venc_mode":	"h265",
			"small_venc_br_ctrl":	"cqp",
			"small_venc_Qfixed":	30,
			"small_venc_Qmin":	15,
			"small_venc_Qmax":	40,
			"small_venc_nPframes":	9,
			"small_venc_mbps":	2,
			"en_large_video":	true,
			"large_video_width":	4056,
			"large_video_height":	3040,
			"large_venc_mode":	"h265",
			"large_venc_br_ctrl":	"cqp",
			"large_venc_Qfixed":	38,
			"large_venc_Qmin":	15,
			"large_venc_Qmax":	50,
			"large_venc_nPframes":	29,
			"large_venc_mbps":	30,
			"en_snapshot":	true,
			"en_snapshot_width":	4056,
			"en_snapshot_height":	3040,
			"exif_focal_length":	3.0999999046325684,
			"exif_focal_length_in_35mm_format":	17,
			"exif_fnumber":	1.2400000095367432,
			"ae_mode":	"isp"
		}]
}

Also, related question, where can I do vertical and horizontal flipping of the image?

@zauberflote1 Thanks! I tried doing this and the voxl camera server and streamer services start up just fine, but it is so overexposed everything is basically white. It seems like my changes to the libmodal exposure settings have no effect on this which makes me think I'm missing something (I'm restarting the camera server and streamer after saving my conf file changes).

I would expect this to give extremely dark pictures, but it's just as white as the defaults

"ae_desired_msv":       10,
"exposure_min_us":      20,
"exposure_max_us":      30,
"gain_min":     0,
"gain_max":     30,
"exposure_soft_min_us": 0,
"ae_filter_alpha":      0.600000023841858,
"ae_ignore_fraction":   0.20000000298023224,
"ae_slope":     0.05000000074505806,
"ae_exposure_period":   1,
"ae_gain_period":       1