AR0144 RGB output on VOXL2
-
I am working on some examples of how to use the GPU with images received from the camera server via MPA (raw bayer, yuv, rgb).
I just wanted to clarify your use case.
- Currently, the camera server receives RAW8 bayer image from AR0144 camera (it can be RAW10/12 but it is not yet working reliably, different story)
- the camera server then uses CPU to de-bayer the image into RGB
- RGB is then sent out via MPA and can be viewed by voxl-portal (which supports RGB) or can be encoded via
voxl-streamer, which also supports RGB input.
Now, in order to do any image stitching, the following would need to be done in the client application
- subscribe the the multiple camera streams (whether bayer or RGB)
- load camera calibration (intrinsic) for each camera
- load extrinsic parameters for each camera
- load LSC (lens shading correction) tables for each camera (or could be the same for all, close enough)
- for simplicity, lets assume that there is one large output buffer is allocated for all cameras to be pasted into (panorama image). and when individual images come in, the job of the app is to do the following:
- apply LSC corrections to the whole image
- (optional) perform white balance corrections (to be consistent across all cameras) (would need some analysis across all images)
- undistort the image according to the fisheye calibration params
- project the image according to the extrinsics calibration into the stitched image
this would ignore the time synchronization aspect for now and basically as each image comes in, the stitched image is updated.
Also, for your algorithm development, do you work with RGB image or YUV?
I am planning to share something simple first, which will subscribe to the 3 or 4 AR0144 cameras, load some intrinsics and extrinsics and overlay the images in a larger image (without explicitly calibrating the extrinsics). Then we can go from there.
Alex
Yes, what you have described for the client application is what I am trying to design. Ideally, I would have time synchronization so that frames always line up, but if latency isn't too big of an issue, this should work fine. I was planning on working with the RGB image, but I'm open to using YUV if it offers a noticeable reduction in latency. Also, to clarify my current setup, I have 4 AR0144 cameras placed at 90-degree angles to one another, so there is plenty of overlap between adjacent cameras for stitching.
A simple example would be extremely useful! Thanks for your help!
-
A Alex Kushleyev referenced this topic on
-
-
-
-
Reading along? Create a free ModalAI Forum account to join in.
With an account you can reply, ask your own question, get an email when a ModalAI engineer answers, and mark the reply that solved it. Your place in each thread is saved between visits.
Questions about VOXL, Flight Core, ESCs and ModalAI drones are answered here by the engineers who build them.
Register Login