ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Fisheye stereo

    Image Sensors
    4
    10
    272
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      david.moro
      last edited by

      I came across this SKU earlier today: https://www.modalai.com/products/m0072?variant=40414007132211

      I have two questions. First, can this sensor substitute the standard tracking camera? It looks like the specs are the same. And the shorter flex cable would be very helpful in one of our integrations. Secondly, can you share any details on how to use it for stereo depth? How does it perform compared with the standard camera? How hard is it to calibrate?

      Thanks!

      ModeratorM 1 Reply Last reply Reply Quote 0
      • ModeratorM
        Moderator ModalAI Team @david.moro
        last edited by

        @david-moro

        Yes, the M0072 image sensor module can be used with the M0010 stereo flex cable.

        The fisheye may be more difficult to calibrate, but will otherwise work in the same way as the more narrow lens.

        See more here: https://docs.modalai.com/M0072/#voxl-2-integration

        D 1 Reply Last reply Reply Quote 0
        • D
          david.moro @Moderator
          last edited by

          @Moderator thanks for the information.

          Could one of the two M0072 be used on the VIO pipeline at the same time? I am thinking it would be great to have VIO and DFS from just two cameras.

          1 Reply Last reply Reply Quote 0
          • D
            david.moro
            last edited by

            Is there any documentation on how to calibrate the fisheye camera when used for stereo? Any example of performance (depth vs. baseline, minimum distance, etc.) would be much appreciated too 🙂

            tomT Alex KushleyevA 2 Replies Last reply Reply Quote 0
            • tomT
              tom admin @david.moro
              last edited by

              This post is deleted!
              1 Reply Last reply Reply Quote 0
              • Alex KushleyevA
                Alex Kushleyev ModalAI Team @david.moro
                last edited by

                @david-moro , the calibration procedure itself would be the same as the regular stereo calibration, you would just need to use -f flag with the voxl-calibrate-camera command. https://docs.modalai.com/calibrate-cameras/#stereo-calibration-process

                Unfortunately we don't have any performance metrics comparing the fisheye vs non-fisheye ov7251 stereo setups.

                I can try calibrating the cameras using our standard calibration procedure, just to make sure that it works, if you are interested.

                Regarding using one of the stereo cameras for VIO, yes, that should technically work, but i need to check something - the stereo image is merged (stacked) image that combines two cameras into one image, but QVIO expects a single image with correct dimensions of a single image. Basically, camera server would need to publish a separate image with just left or right in addition to the image with combined stereo. Alternatively, QVIO server could be updated to accept a stereo image. It looks like right now QVIO sever can accept a stereo image (https://gitlab.com/voxl-public/voxl-sdk/services/voxl-qvio-server/-/blob/master/server/main.cpp?ref_type=heads#L609) but a few lines later it checks for image dimensions to match the dims in calibration file (there would be a mismatch because calibration is mono and image is stereo) .. I will need to double check this..

                Alex

                D 1 Reply Last reply Reply Quote 0
                • D
                  david.moro @Alex Kushleyev
                  last edited by

                  @Alex-Kushleyev , thanks for the thorough explanation. If you could do a quick and dirty check, that would be much appreciated!

                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @david.moro
                    last edited by

                    @david-moro , I just confirmed that my concern regarding stereo input to vio is not actually an issue. The meta information for stereo image type actually has width and height equal to width and height of mono image, but the buffer contains two images. So, QVIO will just pick the first image if stereo is provided.

                    I will do a quick test to see how well the fisheye ov7251 calibration works.

                    Alex

                    Alex KushleyevA 1 Reply Last reply Reply Quote 0
                    • Alex KushleyevA
                      Alex Kushleyev ModalAI Team @Alex Kushleyev
                      last edited by

                      @david-moro ,

                      With a few fixes, I was able to get the ov7251 fisheye stereo to work. Mainly needed to tweak the calibration procedure a bit, and add a fix to actually run fisheye stereo calibration instead of regular stereo calibration for the extrinsics. I will commit the fixes to the voxl-camera-calibration repository soon. The DFS part ran fine without any changes, after getting a proper camera calibration, but I will double check if I needed to change anything in DFS.

                      sample outputs:

                      narrow:
                      ov7251_dfs_narrow.png

                      wide (not wearing the same shirt, so the pattern is different 🙂 )
                      ov7251_dfs_wide.png

                      I probably won't be able to run detailed analysis, but it looks like the DFS pipeline is working for fisheye use case. As you probably guessed, the fisheye setup will have shorter range but wider FOV, which can be estimated mathematically.

                      If you decide to go ahead and buy the fisheye cameras and try it, I just wanted to mention that there is still some risk in robustness of performance, since we have not really tested it until today. However, the results are promising.

                      Also, wanted to mention that the OV7251 are very old cameras. We have much better options available now, specifically AR0144 , which is 1280x800 resolution (monochrome), also global shutter. In order to use synchronized AR0144 cameras, you would need to use our new camera front end (M0173). Please see links below:

                      • https://docs.modalai.com/M0173/
                      • https://docs.modalai.com/voxl2-coax-camera-bundles/
                      • the best deal would probably be getting this C28 configuration (M0173 camera front end + dual AR0144 and dual IMX412 hires cameras + all the coax cables) : https://www.modalai.com/products/m0173?variant=48528274489648
                      • you can also buy these components separately, but the bundle seems like a better deal (even though you did not ask for two hires cameras 🙂 )
                      • the M0173 front end will take up camera connectors J6 and J7 on VOXL2 (J8 will still be available)

                      If you want to wait for 1-2 days i can send you a sample output of the AR0144 stereo DFS.

                      Finally, there is another option, which would plug in directly into the stereo flex that you already have for OV7251 : https://docs.modalai.com/M0113/ . It is OV9782 global shutter RGB camera 1280x800 resolution. It is similar to AR0144, although OV9782 is older sensor and we do not recommend for new designs. We do support it though.
                      Here is a sample image from OV9782 stereo pair, so you can get an idea of the difference. This module does not have a fisheye lens, but it is much wider than the narrow OV7251. Also, the higher resolution makes a big difference.
                      ov9782_dfs.png

                      Alex

                      D 1 Reply Last reply Reply Quote 0
                      • D
                        david.moro @Alex Kushleyev
                        last edited by

                        @Alex-Kushleyev , this is fantastic. Thanks so much!

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Powered by NodeBB | Contributors