@Alex-Kushleyev Also just noticed the vertical artifact again. The artifact is only present in the snapshots and not the raw images saved as jpeg - will definitely need those post processing scripts now.
Here is the raw image:
And here is the snapshot:
@Alex-Kushleyev Sorry for the delayed response, but I believe the ISP snapshots are good enough quality for us, but could I still get the scripts to mess around and test with?
Also is the most recent camera server here the version with the auto-auto exposure? If so, what is the parameter to use to activate that?
@Alex-Kushleyev Our application doesn't specifically need the metadata, just figured I should point it out in case it was intended to be there. It may be useful to have shutter speed, ISO, aperture, and date/time though.
Ah yes that seems to be the issue, we are using MISP, preview, and snapshot. It would be useful to have the AE run without us viewing the stream. Here is the conf file:
{
"version": 0.1,
"fsync_en": false,
"fsync_gpio": 109,
"cameras": [{
"type": "boson",
"name": "boson",
"enabled": true,
"camera_id": 0,
"fps": 30,
"en_preview": true,
"en_misp": true,
"preview_width": 640,
"preview_height": 512,
"en_raw_preview": true,
"en_small_video": false,
"en_large_video": false,
"ae_mode": "off",
"en_rotate": false,
"misp_width": 512,
"misp_height": 640,
"misp_venc_enable": true,
"misp_venc_mode": "h264",
"misp_venc_br_ctrl": "cqp",
"misp_venc_Qfixed": 30,
"misp_venc_Qmin": 15,
"misp_venc_Qmax": 50,
"misp_venc_nPframes": 29,
"misp_venc_mbps": 2,
"misp_venc_osd": false,
"misp_awb": "off",
"misp_gamma": 1,
"misp_zoom": 1,
"gain_min": 100,
"gain_max": 100
}, {
"type": "ov64b",
"name": "hires",
"enabled": true,
"camera_id": 1,
"fps": 30,
"en_preview": true,
"en_misp": true,
"preview_width": 9216,
"preview_height": 6912,
"en_raw_preview": true,
"en_small_video": false,
"en_large_video": false,
"en_snapshot": true,
"ae_mode": "isp",
"gain_min": 100,
"gain_max": 100,
"misp_width": 9216,
"misp_height": 6912,
"misp_venc_enable": false,
"misp_venc_mode": "h265",
"misp_venc_br_ctrl": "cqp",
"misp_venc_Qfixed": 38,
"misp_venc_Qmin": 15,
"misp_venc_Qmax": 50,
"misp_venc_nPframes": 29,
"misp_venc_mbps": 30,
"misp_venc_osd": false,
"misp_awb": "auto",
"misp_gamma": 1,
"misp_zoom": 1,
"ae_desired_msv": 75,
"exposure_min_us": 1000,
"exposure_max_us": 1001,
"exposure_soft_min_us": 5000,
"ae_filter_alpha": 0.6,
"ae_ignore_fraction": 0.2,
"ae_slope": 0.1,
"ae_exposure_period": 1,
"ae_gain_period": 1,
"max_request_queue_depth": 6,
"en_snapshot_width": 9216,
"en_snapshot_height": 6912,
"exif_focal_length": 3.1,
"exif_focal_length_in_35mm_format": 17,
"exif_fnumber": 1.24,
"snapshot_jpeg_quality": 90
}]
}
@Alex-Kushleyev I have been doing some tests and the updates to the camera server and mpa tools seem to work great, thank you! What I meant about the metadata not recording is this data only shows up when using the snapshot command, and does not show up when using the jpeg flag in the voxl-record-raw-image command:
Also noticed the auto exposure doesn't seem to settle unless the camera's feed is opened on the voxl-portal. Here are two images I took - voxl-camera-server was running for a considerable time before taking either one.
Before looking at the live stream in the portal:
After looking at the stream in the portal for a few seconds (I could see the exposure settle down)
Didn't mess with voxl-camera-server at all in between taking the two images.
@Alex-Kushleyev Hey Alex, we have been performing some tests and noticed the images we were getting were quite noisy. What are some of the settings that we can change in the voxl-camera-server to reduce the noise? I messed around with the ae_desired_msv setting and gain_max setting. I also noticed that the white balance seems to have a very visible green shift when the exposure is lower (tried to upload an image but got a "request entity too large" error)
Also as a side note, the metadata seems to not be recording when using voxl-record-raw-image -j
@Alex-Kushleyev Thank you!
I was able to get both the full YUV and them in JPEG working. We also noticed that the vertical artifact seemed to disappear in these images. Regarding the resolution we were using when we had the artifacts, we were using 9248 × 6944 on the old drivers.
@Alex-Kushleyev Thank you! This now works great and I can save the raw files as a .bin file. Do you have tips for performing raw-jpg conversion both on and offline? I have also tried to take snapshots with the current setup, by disabling MISP and Raw preview, and enabling snapshot and setting the snapshot dimensions to 9248x6944 (9216x6944, 4624x3472, and 4608x3472 all give the error of unsupported file size). voxl-camera-server seems to start up, then immediately gets killed without an error. Here is the output of the command:
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
Setting MISP AWB to Auto
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
=================================================================
configuration for 2 cameras:
cam #0
name: boson
type: boson
bayer_type: 0
enabled: 1
camera_id: 0
camera_id_second: -1
fps: 30
en_rotate: 0
en_rotate2: 0
en_preview: 1
en_raw_preview: 1
preview_width: 640
preview_height: 512
en_misp: 1
misp_width: 512
misp_height: 640
en_small_video: 0
small_video_width: 640
small_video_height: 480
en_large_video: 0
large_video_width: -1
large_video_height: -1
en_snapshot: 0
snap_width: -1
snap_height: -1
exif_focal_length: 0.000000
exif_focal_len_35mm_format:0
exif_fnumber: 0.000000
ae_mode: off
msv_exposure_min_us: 20
msv_exposure_max_us: 33000
gain_min : 100
gain_max : 100
standby_enabled: 0
decimator: 1
independent_exposure:0
cam #1
name: hires
type: ov64b
bayer_type: 1
enabled: 1
camera_id: 1
camera_id_second: -1
fps: 30
en_rotate: 0
en_rotate2: 0
en_preview: 1
en_raw_preview: 0
preview_width: 9248
preview_height: 6944
en_misp: 0
misp_width: 1280
misp_height: 720
en_small_video: 0
small_video_width: 1024
small_video_height: 768
en_large_video: 0
large_video_width: 3840
large_video_height: 2160
en_snapshot: 1
snap_width: 9248
snap_height: 6944
exif_focal_length: 3.100000
exif_focal_len_35mm_format:17
exif_fnumber: 1.240000
ae_mode: lme_msv
msv_exposure_min_us: 20
msv_exposure_max_us: 33000
gain_min : 100
gain_max : 1600
standby_enabled: 0
decimator: 1
independent_exposure:0
fsync_en: 0
fsync_gpio: 109
=================================================================
thread is locked to cores: 4 5 6 7
connected to mavlink pipe
Connected to cpu-monitor
Starting Camera: boson (id #0)
ModalExposureMSV: initializing for camera name type
gbm_create_device(156): Info: backend name is: msm_drm
MISP Initializing for camera boson
Detected 1 platform(s)
Detected 1 GPU device(s)
Estimated imu dt = 0.000977s
ERROR in json_from_yaml, failed to open file
MISP: Loading intrinsics cal file boson_intrinsics.yml for camera boson, ret -1
WARNING: Lens calibration for camera boson is missing (boson_intrinsics.yml). Using defaults:
Starting Camera: hires (id #1)
ModalExposureMSV: initializing for camera name hires type ov64b
------ voxl-camera-server: Started 2 of 2 cameras
------ voxl-camera-server: Camera server is now running
Killed
Don't necessarily need to use the snapshot command, but we want some way to get JPG images off the voxl.
@Alex-Kushleyev Hi Alex, thank you for the new drivers and the instructions. Unfortunately I am still having issues getting the hires_bayer pipe to appear. I followed all of the instructions in the README, and when shooting at 9216x6944, I get the following error:
ERROR: MISP: Unexpected frame size for camera hires, width 9216, height 6944, stride 11520, alloc 79994880 bytes, calc frame size 79994880
ERROR: Could not find frame size from the raw buffer
Same error I was getting last week but now the allocated and calculated values are the same. Running it at 9248x6944 gives a similar error:
ERROR: MISP: Unexpected frame size for camera hires, width 9248, height 6944, stride 11568, alloc 80330752 bytes, calc frame size 80328192
ERROR: Could not find frame size from the raw buffer
I do see the boson_bayer pipe, and in the portal the camera feed looks great - I believe we were getting a weird preview with the old drivers, but we didn't look too carefully as we are not using the boson yet. Here is the output of voxl-inspect-cam -a:
| Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fps | mbps | format
| boson | 327680 | 640 | 512 | 0.00 | 800 | 295 | 13.2 | 60.0 | 157.3 | RAW8
| boson_bayer | 327680 | 640 | 512 | 0.00 | 0 | 295 | 12.7 | 60.0 | 157.3 | RAW8
| boson_color | 983040 | 640 | 512 | 0.00 | 800 | 295 | 15.6 | 60.0 | 471.9 | RGB
| boson_misp_encoded | 32 | 640 | 512 | 0.00 | 800 | 295 | 17.0 | 60.0 | 0.0 | H264 (P)
| hires_bayer |
| hires_misp_color |
| hires_misp_grey |
Disabling MISP gives the following warning and output of voxl-inspect-cam -a:
------ voxl-camera-server: Camera server is now running
Received RAW10 frame from camera hires, will be converting to RAW8 on cpu
WARNING: preview buffer pool for Cam(hires), Frame(18) has 0 free, skipping request
| Pipe Name | bytes | wide | hgt |exp(ms)| gain | frame id |latency(ms)| fps | mbps | format
| boson | 327680 | 640 | 512 | 0.00 | 800 | 507 | 12.4 | 60.0 | 157.3 | RAW8
| boson_bayer | 327680 | 640 | 512 | 0.00 | 0 | 507 | 10.9 | 60.0 | 157.3 | RAW8
| boson_color | 983040 | 640 | 512 | 0.00 | 800 | 507 | 13.3 | 60.0 | 471.9 | RGB
| boson_misp_encoded | 32 | 640 | 512 | 0.00 | 800 | 507 | 14.4 | 60.0 | 0.0 | H264 (P)
| hires_bayer |
| hires_color |
| hires_grey |64218112 | 9248 | 6944 | 12.06 | 1211 | 16 | 3720.9 | 1.8 | 945.2 | RAW8
@Alex-Kushleyev Some additional info: When we ran at the lower resolution and tried viewing hires_bayer in the voxl portal this error would pop up in the terminal running voxl-camera-server:
ERROR: Could not find frame size from the raw buffer
ERROR: MISP: Unexpected frame size for camera hires, width 4624, height 3472, stride 5792, alloc 20111360 bytes, calc frame size 20109824
ERROR: Could not find frame size from the raw buffer
Thank you for your speedy response
@Alex-Kushleyev we are still having some issues getting the raw images from the hadron. We have updated to SDK 1.6.3. Here is the results when setting the preview resolution to 4624x3472 and disabling MISP (camera server runs, cant get raw image):
Setting MISP AWB to Auto
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
Setting MISP AWB to Auto
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
=================================================================
configuration for 2 cameras:
cam #0
name: boson
type: boson
bayer_type: 0
enabled: 0
camera_id: 0
camera_id_second: -1
fps: 30
en_rotate: 0
en_rotate2: 0
en_preview: 1
en_raw_preview: 1
preview_width: 640
preview_height: 512
en_misp: 0
misp_width: -1
misp_height: -1
en_small_video: 0
small_video_width: 640
small_video_height: 480
en_large_video: 0
large_video_width: -1
large_video_height: -1
en_snapshot: 0
snap_width: -1
snap_height: -1
exif_focal_length: 0.000000
exif_focal_len_35mm_format:0
exif_fnumber: 0.000000
ae_mode: off
msv_exposure_min_us: 20
msv_exposure_max_us: 33000
gain_min : 54
gain_max : 8000
standby_enabled: 0
decimator: 1
independent_exposure:0
cam #1
name: hires
type: ov64b
bayer_type: 1
enabled: 1
camera_id: 1
camera_id_second: -1
fps: 5
en_rotate: 0
en_rotate2: 0
en_preview: 1
en_raw_preview: 1
preview_width: 4624
preview_height: 3472
en_misp: 0
misp_width: 4624
misp_height: 3472
en_small_video: 1
small_video_width: 1024
small_video_height: 768
en_large_video: 0
large_video_width: 3840
large_video_height: 2160
en_snapshot: 0
snap_width: 3840
snap_height: 2160
exif_focal_length: 3.100000
exif_focal_len_35mm_format:17
exif_fnumber: 1.240000
ae_mode: isp
msv_exposure_min_us: 20
msv_exposure_max_us: 33000
gain_min : 100
gain_max : 32000
standby_enabled: 0
decimator: 1
independent_exposure:0
fsync_en: 0
fsync_gpio: 109
=================================================================
thread is locked to cores: 4 5 6 7
connected to mavlink pipe
Connected to cpu-monitor
Skipping Camera: boson, configuration marked disabled
Starting Camera: hires (id #1)
Estimated imu dt = 0.000977s
gbm_create_device(156): Info: backend name is: msm_drm
------ voxl-camera-server: Started 1 of 2 cameras
------ voxl-camera-server: Camera server is now running
Received RAW10 frame from camera hires, will be converting to RAW8 on cpu
Here is what we see when we do use MISP, and full resolution:
existing instance of voxl-camera-server found, attempting to stop it
Setting MISP AWB to Auto
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
Setting MISP AWB to Auto
MISP channels enabled in defaults : 0
MISP channels enabled in config file: 0
=================================================================
configuration for 2 cameras:
cam #0
name: boson
type: boson
bayer_type: 0
enabled: 0
camera_id: 0
camera_id_second: -1
fps: 30
en_rotate: 0
en_rotate2: 0
en_preview: 1
en_raw_preview: 1
preview_width: 640
preview_height: 512
en_misp: 0
misp_width: -1
misp_height: -1
en_small_video: 0
small_video_width: 640
small_video_height: 480
en_large_video: 0
large_video_width: -1
large_video_height: -1
en_snapshot: 0
snap_width: -1
snap_height: -1
exif_focal_length: 0.000000
exif_focal_len_35mm_format:0
exif_fnumber: 0.000000
ae_mode: off
msv_exposure_min_us: 20
msv_exposure_max_us: 33000
gain_min : 54
gain_max : 8000
standby_enabled: 0
decimator: 1
independent_exposure:0
cam #1
name: hires
type: ov64b
bayer_type: 1
enabled: 1
camera_id: 1
camera_id_second: -1
fps: 5
en_rotate: 0
en_rotate2: 0
en_preview: 1
en_raw_preview: 1
preview_width: 9248
preview_height: 6944
en_misp: 1
misp_width: 1920
misp_height: 1080
en_small_video: 1
small_video_width: 1024
small_video_height: 768
en_large_video: 0
large_video_width: 3840
large_video_height: 2160
en_snapshot: 0
snap_width: 3840
snap_height: 2160
exif_focal_length: 3.100000
exif_focal_len_35mm_format:17
exif_fnumber: 1.240000
ae_mode: lme_msv
msv_exposure_min_us: 20
msv_exposure_max_us: 33000
gain_min : 100
gain_max : 32000
standby_enabled: 0
decimator: 1
independent_exposure:0
fsync_en: 0
fsync_gpio: 109
=================================================================
thread is locked to cores: 4 5 6 7
connected to mavlink pipe
Connected to cpu-monitor
Skipping Camera: boson, configuration marked disabled
Starting Camera: hires (id #1)
ERROR: Configure streams failed for camera: 1
ERROR: Failed to configure streams for camera: hires
ERROR: Failed to start camera: hires
ERROR: failed to initialize any cameras
------ voxl-camera-server: Started 0 of 2 cameras
------ voxl-camera-server: Camera server is now running
------ voxl-camera-server: Camera server is now stopping
------ voxl-camera-server: Camera server exited gracefully, returning -1
IMU Manager closing and exiting