voxl-camera-server bugs with snapshot capture for mapping
-
Hi. I was evaluating the voxl-camera-server snapshot feature for the purpose of using the images for othromaps and pointcoud generation. I found multiple things which are not working properly. I am using an external px4 (mro-ctrl-zero-h7) which sends an onboard mavlink stream to a serial port on voxl-mini. Timebased snapshot capture was working fine so I moved to an acutal survey mission outdoors, which is when I discovered none of the images were saved on the voxl. I set up a testbench where I am flying SITL missions on my laptop and use Mavproxy to output the SITL mavlink through USB-TTL adapter to VOXL. To check the snapshot timing information I point the IMX214 camera at my screen which shows the SITL QGC simulation. Here are my findings:
-
If I use the default mavlink datarate and baudrate the snapshot capturing is not reliable. Sometimes no snapshots get captured at all. When I moved to using a custom mavlink stream on px4 which only outputs command messages and GPS info the snapshot feature started working reliably.
-
I can only reliably capture camera images at most at 1HZ. If I increase the camera capture command frequency beyond 1Hz snapshots get lost.
-
GPS Exif image data contains wrong GPS seconds data. Degrees and seconds are correct but seconds is wrong. (SITL GPS is in Zurich and VOXL Exif data claims 37 deg 24' 3.85" N, 121 deg 59' 3.11" W which is Santa Clara )
Here is a link to an archive with the snapshots captured by voxl with the SITL mission file I was running:
https://drive.google.com/file/d/1vA5-F1MUICMC2DUvb5XdrhR2N_YSinao/view?usp=sharing -
-
Hi. I was evaluating the voxl-camera-server snapshot feature for the purpose of using the images for othromaps and pointcoud generation. I found multiple things which are not working properly. I am using an external px4 (mro-ctrl-zero-h7) which sends an onboard mavlink stream to a serial port on voxl-mini. Timebased snapshot capture was working fine so I moved to an acutal survey mission outdoors, which is when I discovered none of the images were saved on the voxl. I set up a testbench where I am flying SITL missions on my laptop and use Mavproxy to output the SITL mavlink through USB-TTL adapter to VOXL. To check the snapshot timing information I point the IMX214 camera at my screen which shows the SITL QGC simulation. Here are my findings:
-
If I use the default mavlink datarate and baudrate the snapshot capturing is not reliable. Sometimes no snapshots get captured at all. When I moved to using a custom mavlink stream on px4 which only outputs command messages and GPS info the snapshot feature started working reliably.
-
I can only reliably capture camera images at most at 1HZ. If I increase the camera capture command frequency beyond 1Hz snapshots get lost.
-
GPS Exif image data contains wrong GPS seconds data. Degrees and seconds are correct but seconds is wrong. (SITL GPS is in Zurich and VOXL Exif data claims 37 deg 24' 3.85" N, 121 deg 59' 3.11" W which is Santa Clara )
Here is a link to an archive with the snapshots captured by voxl with the SITL mission file I was running:
https://drive.google.com/file/d/1vA5-F1MUICMC2DUvb5XdrhR2N_YSinao/view?usp=sharing@AndriiHlyvko Snapshots commands are handled by the voxl-mavcam-manager application. It has it's own mavlink connection to QGC. You can enable debug messages in voxl-mavcam-manager by adding the
-doption to see what commands it is getting. The images will have timestamps based on the system time at the voxl, not of the external flight controller. -
-
@AndriiHlyvko Snapshots commands are handled by the voxl-mavcam-manager application. It has it's own mavlink connection to QGC. You can enable debug messages in voxl-mavcam-manager by adding the
-doption to see what commands it is getting. The images will have timestamps based on the system time at the voxl, not of the external flight controller.@Eric-Katzfey do you know if there a way to run a remote gdb server through adb and then just insert breakpoints in the mavcam manager?
-
@AndriiHlyvko Snapshots commands are handled by the voxl-mavcam-manager application. It has it's own mavlink connection to QGC. You can enable debug messages in voxl-mavcam-manager by adding the
-doption to see what commands it is getting. The images will have timestamps based on the system time at the voxl, not of the external flight controller.@Eric-Katzfey I added my own print statements to voxl-camera manager that prints GPS data it grams from mavlink before it inserts the information as exif. It is correct GPS data:
ANDRII GPS mavlink: lat:37.413543, lon:-121.996550 Camera: hires taking snapshot (destination: /data/snapshots/hires-2024-05-16_21:18:18.jpg) ANDRII GPS mavlink: lat:37.413543, lon:-121.996550 ANDRII grabbed gps: lat:37.413543, lon:-121.996550 Camera: hires writing snapshot to :"/data/snapshots/hires-2024-05-16_21:18:18.jpg" ANDRII GPS mavlink: lat:37.413543, lon:-121.996550But then when I inspect the snapshot image with exiftool I see the information is not correct:
---- ExifTool ---- ExifTool Version Number : 12.40 ---- File ---- File Name : hires-2024-05-16_21:18:18.jpg Directory : ./snapshots File Size : 641 KiB File Modification Date/Time : 2024:05:16 14:18:41-07:00 File Access Date/Time : 2024:05:16 14:18:41-07:00 File Inode Change Date/Time : 2024:05:16 14:18:41-07:00 File Permissions : -rw-r--r-- File Type : JPEG File Type Extension : jpg MIME Type : image/jpeg Exif Byte Order : Big-endian (Motorola, MM) Exif Byte Order : Big-endian (Motorola, MM) Image Width : 4208 Image Height : 3120 Encoding Process : Baseline DCT, Huffman coding Bits Per Sample : 8 Color Components : 3 Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2) ---- EXIF ---- Image Width : 4208 Image Height : 3120 Make : QTI-AA Camera Model Name : QCAM-AA Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Modify Date : 2024:05:16 21:18:18 Y Cb Cr Positioning : Centered Exposure Time : 1/120 F Number : 2.4 Exposure Program : Program AE ISO : 194 Exif Version : 0220 Date/Time Original : 2024:05:16 21:18:18 Create Date : 2024:05:16 21:18:18 Components Configuration : Y, Cb, Cr, - Shutter Speed Value : 1/120 Aperture Value : 2.4 Brightness Value : 3.52 Exposure Compensation : 0 Max Aperture Value : 2.4 Metering Mode : Average Light Source : D65 Flash : Off, Did not fire Focal Length : 2.2 mm Sub Sec Time : 340360 Sub Sec Time Original : 340360 Sub Sec Time Digitized : 340360 Flashpix Version : 0100 Color Space : sRGB Exif Image Width : 4208 Exif Image Height : 3120 Interoperability Index : R98 - DCF basic file (sRGB) Interoperability Version : 0100 Sensing Method : Not defined Scene Type : Directly photographed Exposure Mode : Auto White Balance : Auto Focal Length In 35mm Format : 2 mm Scene Capture Type : Standard GPS Latitude Ref : North GPS Latitude : 37 deg 24' 1.00" GPS Longitude Ref : West GPS Longitude : 121 deg 59' 1.48" GPS Altitude : 1 m Compression : JPEG (old-style) Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Thumbnail Offset : 986 Thumbnail Length : 15135 Image Width : 4208 Image Height : 3120 Make : QTI-AA Camera Model Name : QCAM-AA Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Modify Date : 2024:05:16 21:18:18 Y Cb Cr Positioning : Centered Exposure Time : 1/120 F Number : 2.4 Exposure Program : Program AE ISO : 194 Exif Version : 0220 Date/Time Original : 2024:05:16 21:18:18 Create Date : 2024:05:16 21:18:18 Components Configuration : Y, Cb, Cr, - Shutter Speed Value : 1/120 Aperture Value : 2.4 Brightness Value : 3.52 Exposure Compensation : 0 Max Aperture Value : 2.4 Metering Mode : Average Light Source : D65 Flash : Off, Did not fire Focal Length : 2.2 mm Sub Sec Time : 340360 Sub Sec Time Original : 340360 Sub Sec Time Digitized : 340360 Flashpix Version : 0100 Color Space : sRGB Exif Image Width : 4208 Exif Image Height : 3120 Interoperability Index : R98 - DCF basic file (sRGB) Interoperability Version : 0100 Sensing Method : Not defined Scene Type : Directly photographed Exposure Mode : Auto White Balance : Auto Focal Length In 35mm Format : 2 mm Scene Capture Type : Standard Exif Image Width : 320 Exif Image Height : 240 Compression : JPEG (old-style) Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Thumbnail Offset : 17013 Thumbnail Length : 15135 Thumbnail Image : (Binary data 15135 bytes, use -b option to extract) ---- Composite ---- Aperture : 2.4 Image Size : 4208x3120 Megapixels : 13.1 Scale Factor To 35 mm Equivalent: 0.9 Shutter Speed : 1/120 Create Date : 2024:05:16 21:18:18.340360 Date/Time Original : 2024:05:16 21:18:18.340360 Modify Date : 2024:05:16 21:18:18.340360 GPS Latitude : 37 deg 24' 1.00" N GPS Longitude : 121 deg 59' 1.48" W Circle Of Confusion : 0.033 mm Field Of View : 167.3 deg Focal Length : 2.2 mm (35 mm equivalent: 2.0 mm) GPS Position : 37 deg 24' 1.00" N, 121 deg 59' 1.48" W Hyperfocal Distance : 0.06 m Light Value : 8.5 -
@Eric-Katzfey I added my own print statements to voxl-camera manager that prints GPS data it grams from mavlink before it inserts the information as exif. It is correct GPS data:
ANDRII GPS mavlink: lat:37.413543, lon:-121.996550 Camera: hires taking snapshot (destination: /data/snapshots/hires-2024-05-16_21:18:18.jpg) ANDRII GPS mavlink: lat:37.413543, lon:-121.996550 ANDRII grabbed gps: lat:37.413543, lon:-121.996550 Camera: hires writing snapshot to :"/data/snapshots/hires-2024-05-16_21:18:18.jpg" ANDRII GPS mavlink: lat:37.413543, lon:-121.996550But then when I inspect the snapshot image with exiftool I see the information is not correct:
---- ExifTool ---- ExifTool Version Number : 12.40 ---- File ---- File Name : hires-2024-05-16_21:18:18.jpg Directory : ./snapshots File Size : 641 KiB File Modification Date/Time : 2024:05:16 14:18:41-07:00 File Access Date/Time : 2024:05:16 14:18:41-07:00 File Inode Change Date/Time : 2024:05:16 14:18:41-07:00 File Permissions : -rw-r--r-- File Type : JPEG File Type Extension : jpg MIME Type : image/jpeg Exif Byte Order : Big-endian (Motorola, MM) Exif Byte Order : Big-endian (Motorola, MM) Image Width : 4208 Image Height : 3120 Encoding Process : Baseline DCT, Huffman coding Bits Per Sample : 8 Color Components : 3 Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2) ---- EXIF ---- Image Width : 4208 Image Height : 3120 Make : QTI-AA Camera Model Name : QCAM-AA Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Modify Date : 2024:05:16 21:18:18 Y Cb Cr Positioning : Centered Exposure Time : 1/120 F Number : 2.4 Exposure Program : Program AE ISO : 194 Exif Version : 0220 Date/Time Original : 2024:05:16 21:18:18 Create Date : 2024:05:16 21:18:18 Components Configuration : Y, Cb, Cr, - Shutter Speed Value : 1/120 Aperture Value : 2.4 Brightness Value : 3.52 Exposure Compensation : 0 Max Aperture Value : 2.4 Metering Mode : Average Light Source : D65 Flash : Off, Did not fire Focal Length : 2.2 mm Sub Sec Time : 340360 Sub Sec Time Original : 340360 Sub Sec Time Digitized : 340360 Flashpix Version : 0100 Color Space : sRGB Exif Image Width : 4208 Exif Image Height : 3120 Interoperability Index : R98 - DCF basic file (sRGB) Interoperability Version : 0100 Sensing Method : Not defined Scene Type : Directly photographed Exposure Mode : Auto White Balance : Auto Focal Length In 35mm Format : 2 mm Scene Capture Type : Standard GPS Latitude Ref : North GPS Latitude : 37 deg 24' 1.00" GPS Longitude Ref : West GPS Longitude : 121 deg 59' 1.48" GPS Altitude : 1 m Compression : JPEG (old-style) Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Thumbnail Offset : 986 Thumbnail Length : 15135 Image Width : 4208 Image Height : 3120 Make : QTI-AA Camera Model Name : QCAM-AA Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Modify Date : 2024:05:16 21:18:18 Y Cb Cr Positioning : Centered Exposure Time : 1/120 F Number : 2.4 Exposure Program : Program AE ISO : 194 Exif Version : 0220 Date/Time Original : 2024:05:16 21:18:18 Create Date : 2024:05:16 21:18:18 Components Configuration : Y, Cb, Cr, - Shutter Speed Value : 1/120 Aperture Value : 2.4 Brightness Value : 3.52 Exposure Compensation : 0 Max Aperture Value : 2.4 Metering Mode : Average Light Source : D65 Flash : Off, Did not fire Focal Length : 2.2 mm Sub Sec Time : 340360 Sub Sec Time Original : 340360 Sub Sec Time Digitized : 340360 Flashpix Version : 0100 Color Space : sRGB Exif Image Width : 4208 Exif Image Height : 3120 Interoperability Index : R98 - DCF basic file (sRGB) Interoperability Version : 0100 Sensing Method : Not defined Scene Type : Directly photographed Exposure Mode : Auto White Balance : Auto Focal Length In 35mm Format : 2 mm Scene Capture Type : Standard Exif Image Width : 320 Exif Image Height : 240 Compression : JPEG (old-style) Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Thumbnail Offset : 17013 Thumbnail Length : 15135 Thumbnail Image : (Binary data 15135 bytes, use -b option to extract) ---- Composite ---- Aperture : 2.4 Image Size : 4208x3120 Megapixels : 13.1 Scale Factor To 35 mm Equivalent: 0.9 Shutter Speed : 1/120 Create Date : 2024:05:16 21:18:18.340360 Date/Time Original : 2024:05:16 21:18:18.340360 Modify Date : 2024:05:16 21:18:18.340360 GPS Latitude : 37 deg 24' 1.00" N GPS Longitude : 121 deg 59' 1.48" W Circle Of Confusion : 0.033 mm Field Of View : 167.3 deg Focal Length : 2.2 mm (35 mm equivalent: 2.0 mm) GPS Position : 37 deg 24' 1.00" N, 121 deg 59' 1.48" W Hyperfocal Distance : 0.06 m Light Value : 8.5@AndriiHlyvko lat:37.413543, lon:-121.996550 is 37°24'48.8"N 121°59'47.6"W and not 37 deg 24' 1.00" N, 121 deg 59' 1.48" W
-
@AndriiHlyvko Snapshots commands are handled by the voxl-mavcam-manager application. It has it's own mavlink connection to QGC. You can enable debug messages in voxl-mavcam-manager by adding the
-doption to see what commands it is getting. The images will have timestamps based on the system time at the voxl, not of the external flight controller.@Eric-Katzfey Idk if you are still interested but I fixed the bug in the voxl-camera-server. It turnes out whoever wrote it didn't do research on how exif format works. The exif data that was being appended to the image was written in intel byteorder while the camera uses motorolla byteorder. I think your HAL driver needs to check the byteordering when the camera is being set up. But here is the fix that works for the IMX214 camera:
// degrees, min, seconds is each a Radional // a rational is two uint32 numbers entry = create_tag(exif, EXIF_IFD_GPS, (ExifTag)EXIF_TAG_GPS_LATITUDE, 3*sizeof(ExifRational)); entry->format = EXIF_FORMAT_RATIONAL; entry->components = 3; ExifLong degrees_lat = static_cast<ExifLong>(gps_grabbed_info.latitude); double fractional = (gps_grabbed_info.latitude - degrees_lat); ExifLong minutes_lat = static_cast<ExifLong>(fractional*60); fractional = (fractional*60) - minutes_lat; double seconds = (fractional * 60); ExifRational degrees_r = { degrees_lat*10, 10 }; ExifRational minutes_r = { minutes_lat, 1 }; ExifRational seconds_r = { static_cast<ExifLong>(seconds * 1000000), 1000000 }; // Increased precision for seconds //ExifRational seconds_r = { static_cast<ExifLong>(seconds), 1 }; //ExifRational seconds_r = { 3375UL, 100UL }; printf("Seconds int:%u\n", seconds_r.numerator); printf("Seconds double:%lf\n", seconds); exif_set_rational(entry->data, EXIF_BYTE_ORDER_MOTOROLA, degrees_r); exif_set_rational(entry->data + sizeof(ExifRational), EXIF_BYTE_ORDER_MOTOROLA, minutes_r); exif_set_rational(entry->data + 2 * sizeof(ExifRational), EXIF_BYTE_ORDER_MOTOROLA, seconds_r); -
@Eric-Katzfey Idk if you are still interested but I fixed the bug in the voxl-camera-server. It turnes out whoever wrote it didn't do research on how exif format works. The exif data that was being appended to the image was written in intel byteorder while the camera uses motorolla byteorder. I think your HAL driver needs to check the byteordering when the camera is being set up. But here is the fix that works for the IMX214 camera:
// degrees, min, seconds is each a Radional // a rational is two uint32 numbers entry = create_tag(exif, EXIF_IFD_GPS, (ExifTag)EXIF_TAG_GPS_LATITUDE, 3*sizeof(ExifRational)); entry->format = EXIF_FORMAT_RATIONAL; entry->components = 3; ExifLong degrees_lat = static_cast<ExifLong>(gps_grabbed_info.latitude); double fractional = (gps_grabbed_info.latitude - degrees_lat); ExifLong minutes_lat = static_cast<ExifLong>(fractional*60); fractional = (fractional*60) - minutes_lat; double seconds = (fractional * 60); ExifRational degrees_r = { degrees_lat*10, 10 }; ExifRational minutes_r = { minutes_lat, 1 }; ExifRational seconds_r = { static_cast<ExifLong>(seconds * 1000000), 1000000 }; // Increased precision for seconds //ExifRational seconds_r = { static_cast<ExifLong>(seconds), 1 }; //ExifRational seconds_r = { 3375UL, 100UL }; printf("Seconds int:%u\n", seconds_r.numerator); printf("Seconds double:%lf\n", seconds); exif_set_rational(entry->data, EXIF_BYTE_ORDER_MOTOROLA, degrees_r); exif_set_rational(entry->data + sizeof(ExifRational), EXIF_BYTE_ORDER_MOTOROLA, minutes_r); exif_set_rational(entry->data + 2 * sizeof(ExifRational), EXIF_BYTE_ORDER_MOTOROLA, seconds_r);@AndriiHlyvko thanks! Feel free to submit a merge request. Otherwise, we'll look to pull in manually
-
@AndriiHlyvko thanks! Feel free to submit a merge request. Otherwise, we'll look to pull in manually
@Moderator sounds good I will try to create a merge request for this sometime early next week
-
@Moderator sounds good I will try to create a merge request for this sometime early next week
@Moderator Whats the procedure for creating merge requests that are from someone outside of ModalAI? The repo doesn't allow pushing a new branch. I also tried to send an email a link generated by gitlab and that didn't work.
-
@Moderator Whats the procedure for creating merge requests that are from someone outside of ModalAI? The repo doesn't allow pushing a new branch. I also tried to send an email a link generated by gitlab and that didn't work.
-
@AndriiHlyvko Can you try forking the repo, pushing your changes to your fork, then opening a merge request from your fork into the dev branch?
@tom cool. I created an MR with wrong source branch by mistake so just delete that one. But this one is the right one: https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/merge_requests/42
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login