ModalAI Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    VOXL 2 M0135 Dual Camera Connections with M0107 and M0149 Cameras

    Ask your questions right here!
    3
    30
    1772
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • WineVirusW
      WineVirus
      last edited by

      I'm pretty new to wiring/connectors/datasheets, so excuse me if my question seems a bit basic;

      I wanted to ask about connections for sensors. I have an M0107 and an M0149 sensor, the M0135 chip, on a VOXL 2 Mini. SDK 1.1.1. From what I've found, the M0107 sensor plugs into the JU port on the Dual Sensor chip, which plugs into the J7 port on the VOXL 2 Mini. Thus, the M0149 sensor plugs into the JL port on the M0135 chip. I have no idea if I'm correct. I'm doing my best to follow docs but I've hit a wall. Can someone confirm what the correct connections are?

      Thanks in advance!

      IMG_1225 (1) (1).jpg

      Alex KushleyevA 1 Reply Last reply Reply Quote 0
      • Alex KushleyevA
        Alex Kushleyev ModalAI Team @WineVirus
        last edited by

        @WineVirus ,

        Here is one way of connecting these cameras via M0135 adapter:

        • make sure the power to VOXL2 is off before connecting / disconnecting cameras
        • for all connectors, look for pin 1 or white dot and match up the mating connectors so that pin 1 connects to pin 1
        • connect AR0144 (M0149) to M0135 JL (camera slot ID 2)
        • connect IMX412 (M0107) to M0135 JU (camera slot ID 3)
        • connect M0135 with the two cameras to VOXL mini J7

        Software setup:

        • since you are using SDK 1.1.1 and we have updated AR0144 driver since (based on this chanelog ), download the latest driver for camera slot #2 (com.qti.sensormodule.ar0144_2.bin) from here and put it in /usr/lib/camera/
        • copy the IMX412 driver cp /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_3.bin /usr/lib/camera/

        Then if you run voxl-camera-server -l, it should detect two cameras and print our their resolutions. (if needed, stop voxl-camera-server service that may run in the background : systemctl stop voxl-camera-server

        Before you can use the cameras, you need to update /etc/modalai/voxl-camera-server.conf to look something like this:

        {
        	"version":	0.1,
        	"cameras":	[{
        			"type":	"ar0144",
        			"name":	"tracking",
        			"enabled":	true,
        			"camera_id":	0,
        			"fps":	30,
        			"en_rotate":	false,
        			"en_preview":	true,
        			"preview_width":	1280,
        			"preview_height":	800,
        			"ae_mode":	"lme_msv",
        			"ae_desired_msv":	30,
        			"exposure_soft_min_us":	5000,
        			"ae_filter_alpha":	0.6,
        			"ae_ignore_fraction":	0.2,
        			"ae_slope":	0.05,
        			"ae_exposure_period":	1,
        			"ae_gain_period":	1,
        			"en_raw_preview":	true,
        		}, {
        			"type":	"imx412",
        			"name":	"hires",
        			"enabled":	true,
        			"camera_id":	1,
        			"fps":	30,
        			"en_rotate":	false,
        			"en_preview":	false,
        			"preview_width":	640,
        			"preview_height":	480,
        			"en_small_video":	false,
        			"small_video_width":	1280,
        			"small_video_height":	720,
        			"small_venc_mode":	"h265",
        			"small_venc_br_ctrl":	"cqp",
        			"small_venc_Qfixed":	30,
        			"small_venc_Qmin":	15,
        			"small_venc_Qmax":	40,
        			"small_venc_nPframes":	9,
        			"small_venc_mbps":	2,
        			"en_large_video":	true,
        			"large_video_width":	1920,
        			"large_video_height":	1080,
        			"large_venc_mode":	"h265",
        			"large_venc_br_ctrl":	"cqp",
        			"large_venc_Qfixed":	38,
        			"large_venc_Qmin":	15,
        			"large_venc_Qmax":	50,
        			"large_venc_nPframes":	29,
        			"large_venc_mbps":	5,
        			"en_snapshot":	false,
        			"en_snapshot_width":	1280,
        			"en_snapshot_height":	720,
        			"ae_mode":	"isp",
        			"en_raw_preview":	false
        		}]
        }
        

        Please follow these steps and let me know if you run into any issues. After the config file update, you should be able to start voxl camera server. You can do it in the foreground (assuming the background service is stopped), just run voxl-camera-server in the voxl2 terminal.

        A quick test to see if the cameras are running, you can do:

        voxl-inspect-cam -a
        

        should print all available camera sterams

        WineVirusW 2 Replies Last reply Reply Quote 0
        • WineVirusW
          WineVirus @Alex Kushleyev
          last edited by

          This post is deleted!
          1 Reply Last reply Reply Quote 0
          • WineVirusW
            WineVirus @Alex Kushleyev
            last edited by

            @Alex-Kushleyev voxl-camera-server -l outputs

            voxl2-mini:/$ voxl-camera-server -l
            DEBUG:   Attempting to open the hal module
            DEBUG:   SUCCESS: Camera module opened on attempt 0
            DEBUG:   ----------- Number of cameras: 0
            
            DEBUG:   Note: This list comes from the HAL module and may not be indicative
            DEBUG:          of configurations that have full pipelines
            
            DEBUG:   Number of cameras: 0
            

            I also tried voxl-camera-server -s which gave me

            voxl2-mini:/$ voxl-camera-server -s
            GPS server Connected
            ERROR:   Did not detect any cameras plugged in
            ERROR:   Failed to get valid debug configuration 
            

            I'm sure my connections are correct as detailed above (pin 1 on pin 1, correct JL/JU, and on the J7 port). Could it be a bad connector?

            Alex KushleyevA 1 Reply Last reply Reply Quote 0
            • Alex KushleyevA
              Alex Kushleyev ModalAI Team @WineVirus
              last edited by

              @WineVirus , can you please provide a picture of the connected cameras to VOXL2 and contents of /usr/lib/camera on your VOXL2?

              WineVirusW 1 Reply Last reply Reply Quote 0
              • WineVirusW
                WineVirus @Alex Kushleyev
                last edited by WineVirus

                @Alex-Kushleyev Here's the output of ./adb shell ls -la /usr/lib/camera . I'll take a picture of the connections when I can. Also, I'm using the VOXL 2 MINI, don't know if that makes any difference.

                total 4664 drwxr-xr-x 3 root root 4096 Mar 2 13:10 . drwxr-xr-x 69 root root 20480 Mar 2 12:59 .. -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.eeprom.cmk_imx577.so -rwxr-xr-x 1 root root 9904 Oct 26 2023 com.qti.sensor.ar0144.so -rwxr-xr-x 1 root root 5816 Oct 26 2023 com.qti.sensor.cmk_imx577.so -rwxr-xr-x 1 root root 5816 Oct 26 2023 com.qti.sensor.cmk_imx577_flip.so -rwxr-xr-x 1 root root 9904 Oct 26 2023 com.qti.sensor.imx214.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.imx214_flip.so -rwxr-xr-x 1 root root 9904 Oct 26 2023 com.qti.sensor.imx335.so -rwxr-xr-x 1 root root 5808 Oct 26 2023 com.qti.sensor.imx377.so -rwxr-xr-x 1 root root 9904 Oct 26 2023 com.qti.sensor.imx678.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.imx678_flip.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.imx678_mod.so -rwxr-xr-x 1 root root 5808 Oct 26 2023 com.qti.sensor.irs1645.so -rwxr-xr-x 1 root root 9904 Oct 26 2023 com.qti.sensor.ov7251.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov7251_front_left.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov7251_front_left_flip.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov7251_front_right.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov7251_front_right_flip.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov7251_fsin.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov7251_fsout.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov7251_rear_left.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov7251_rear_left_flip.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov7251_rear_right.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov7251_rear_right_flip.so -rwxr-xr-x 1 root root 9904 Oct 26 2023 com.qti.sensor.ov9782.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov9782_front_left.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov9782_front_left_flip.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov9782_front_right.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov9782_front_right_flip.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov9782_rear_left.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov9782_rear_left_flip.so -rwxr-xr-x 1 root root 9912 Oct 26 2023 com.qti.sensor.ov9782_rear_right.so -rwxr-xr-x 1 root root 9920 Oct 26 2023 com.qti.sensor.ov9782_rear_right_flip.so -rw-rw-rw- 1 root root 31984 Mar 26 2024 com.qti.sensormodule.ar0144_2.bin -rw-r--r-- 1 root root 578576 Mar 2 13:12 com.qti.sensormodule.imx412_flip_3.bin -rw-r--r-- 1 root root 142472 Mar 2 13:12 com.qti.sensormodule.ov7251_2.bin -rw-r--r-- 1 root root 957344 Oct 26 2023 com.qti.tuned.cmk_imx577.bin -rw-r--r-- 1 root root 835464 Oct 26 2023 com.qti.tuned.cmk_ov9282.bin -rw-r--r-- 1 root root 931864 Oct 26 2023 com.qti.tuned.default.bin -rw-r--r-- 1 root root 882376 Oct 26 2023 com.qti.tuned.sony_imx335.bin drwxr-xr-x 2 root root 4096 Oct 26 2023 components

                Alex KushleyevA 1 Reply Last reply Reply Quote 0
                • Alex KushleyevA
                  Alex Kushleyev ModalAI Team @WineVirus
                  last edited by

                  @WineVirus

                  I will test your camera configuration on voxl2 mini running sdk 1.1.1 and will follow up soon.

                  If you have a picture that will be helpful to confirm correct connections.

                  Alex KushleyevA 1 Reply Last reply Reply Quote 0
                  • Alex KushleyevA
                    Alex Kushleyev ModalAI Team @Alex Kushleyev
                    last edited by

                    Hi, I just connected the two cameras to voxl-2 mini running SDK 1.1.1.

                    For ar0144, please use the latest sensormodule file from this location . The one that comes with SDK 1.1.1 has stability issues.

                    #from your PC
                    adb push ~/Downloads/com.qti.sensormodule.ar0144_2.bin /usr/lib/camera/
                    
                    #now on VOXL2 mini
                    voxl2-mini:/$ cp /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_3.bin /usr/lib/camera/
                    
                    voxl2-mini:/$ voxl-camera-server -l
                    DEBUG:   Attempting to open the hal module
                    DEBUG:   SUCCESS: Camera module opened on attempt 0
                    DEBUG:   ----------- Number of cameras: 2
                    
                    DEBUG:   Cam idx: 0, Cam slot: 2, Slave Address: 0x0030, Sensor Id: 0x0356
                    DEBUG:   Cam idx: 1, Cam slot: 3, Slave Address: 0x0034, Sensor Id: 0x0577
                    DEBUG:   Note: This list comes from the HAL module and may not be indicative
                    DEBUG:   	of configurations that have full pipelines
                    
                    DEBUG:   Number of cameras: 2
                    ...
                    

                    The /etc/modalai/voxl-camera-server.conf should look like this (a good starting point):

                    {
                    	"version":	0.1,
                    	"cameras":	[{
                    			"type":	"ar0144",
                    			"name":	"tracking",
                    			"enabled":	true,
                    			"camera_id":	0,
                    			"fps":	30,
                    			"en_rotate":	false,
                    			"en_preview":	true,
                    			"preview_width":	1280,
                    			"preview_height":	800,
                    			"en_raw_preview":	true,
                    			"ae_mode":	"lme_msv",
                    			"ae_desired_msv":	60,
                    			"exposure_min_us":	20,
                    			"exposure_max_us":	33000,
                    			"gain_min":	54,
                    			"gain_max":	8000,
                    			"exposure_soft_min_us":	5000,
                    			"ae_filter_alpha":	0.600000023841858,
                    			"ae_ignore_fraction":	0.20000000298023224,
                    			"ae_slope":	0.05000000074505806,
                    			"ae_exposure_period":	1,
                    			"ae_gain_period":	1
                    		}, {
                    			"type":	"imx412",
                    			"name":	"hires",
                    			"enabled":	true,
                    			"camera_id":	1,
                    			"fps":	30,
                    			"en_preview":	false,
                    			"preview_width":	640,
                    			"preview_height":	480,
                    			"en_raw_preview":	false,
                    			"en_small_video":	true,
                    			"small_video_width":	1280,
                    			"small_video_height":	800,
                    			"small_venc_mode":	"h265",
                    			"small_venc_br_ctrl":	"cqp",
                    			"small_venc_Qfixed":	30,
                    			"small_venc_Qmin":	15,
                    			"small_venc_Qmax":	40,
                    			"small_venc_nPframes":	9,
                    			"small_venc_mbps":	2,
                    			"en_large_video":	true,
                    			"large_video_width":	3840,
                    			"large_video_height":	2160,
                    			"large_venc_mode":	"h265",
                    			"large_venc_br_ctrl":	"cqp",
                    			"large_venc_Qfixed":	38,
                    			"large_venc_Qmin":	15,
                    			"large_venc_Qmax":	50,
                    			"large_venc_nPframes":	29,
                    			"large_venc_mbps":	30,
                    			"en_snapshot":	false,
                    			"en_snapshot_width":	4056,
                    			"en_snapshot_height":	3040,
                    			"ae_mode":	"isp"
                    		}]
                    }
                    

                    then you can start voxl-camera-server in regular mode (without -l), it should run.

                    use voxl-inspect-cam -a to check video streams:

                    |           Pipe Name |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  | format
                    |   hires_large_color |12441600 | 3840 | 2160 | 33.23 | 1125 |      566 |     78.3  | 29.9 | 2974.3 | NV12
                    | hires_large_encoded |    4893 | 3840 | 2160 | 33.23 | 1125 |      566 |     89.2  | 29.9 |    1.3 | H265 (P)   
                    |    hires_large_grey | 8294400 | 3840 | 2160 | 33.23 | 1125 |      567 |     70.6  | 29.9 | 1982.9 | RAW8
                    |   hires_small_color | 1536000 | 1280 |  800 | 33.23 | 1125 |      568 |     57.3  | 29.9 |  367.2 | NV12
                    | hires_small_encoded |    2710 | 1280 |  800 | 33.23 | 1125 |      568 |     60.9  | 29.9 |    1.8 | H265 (P)   
                    |    hires_small_grey | 1024000 | 1280 |  800 | 33.23 | 1125 |      568 |     55.5  | 29.9 |  244.8 | RAW8
                    |            tracking | 1024000 | 1280 |  800 |  7.04 |  665 |     8043 |     20.7  | 30.0 |  245.6 | RAW8
                    

                    Then you can use voxl-portal to view images (assuming you have a wired or wifi adapter plugged into your voxl2-mini:
                    (if voxl portal is not enabled, you can enable it):

                    systemctl enable voxl-portal
                    systemctl start voxl-portal
                    

                    Then go to web browser and enter your voxl2-mini's ip to view voxl-portal

                    Alex KushleyevA WineVirusW 3 Replies Last reply Reply Quote 0
                    • Alex KushleyevA
                      Alex Kushleyev ModalAI Team @Alex Kushleyev
                      last edited by

                      images of my setup attached

                      IMG_3880.jpg
                      IMG_3881.jpg

                      1 Reply Last reply Reply Quote 0
                      • WineVirusW
                        WineVirus @Alex Kushleyev
                        last edited by WineVirus

                        @Alex-Kushleyev Here's what I'm seeing: bc33809b-869c-4beb-a235-a691d3cde4be-image.png

                        Here's the wiring; It looks the same as yours. IMG_1311 (1).jpg IMG_1312 (1).jpg

                        Alex KushleyevA 1 Reply Last reply Reply Quote 0
                        • Alex KushleyevA
                          Alex Kushleyev ModalAI Team @WineVirus
                          last edited by

                          @WineVirus i believe you have M0135 plugged into J6 instead of J7 like I do. Take a look at my photo again.

                          WineVirusW 1 Reply Last reply Reply Quote 0
                          • WineVirusW
                            WineVirus @Alex Kushleyev
                            last edited by

                            @Alex-Kushleyev Ahh, yeah. My board is upside down, so I got turned around.

                            I'm now detecting one camera, but both are plugged in. I'm not sure which one this is?

                            voxl2-mini:/$ voxl-camera-server -l
                            DEBUG:   Attempting to open the hal module
                            DEBUG:   SUCCESS: Camera module opened on attempt 0
                            DEBUG:   ----------- Number of cameras: 1
                            
                            DEBUG:   Cam idx: 0, Cam slot: 2, Slave Address: 0x0030, Sensor Id: 0x0356
                            DEBUG:   Note: This list comes from the HAL module and may not be indicative
                            DEBUG:          of configurations that have full pipelines
                            
                            DEBUG:   Number of cameras: 1  
                            

                            Also, in terms of updating the config - is it a push command, like the ar0144 driver? Not sure what command to use in ADB.

                            Alex KushleyevA 1 Reply Last reply Reply Quote 0
                            • Alex KushleyevA
                              Alex Kushleyev ModalAI Team @WineVirus
                              last edited by

                              @WineVirus please double check that you have com.qti.sensormodule.imx412_3.bin in /usr/lib/camera/ and also double check the orientation of the flex cable coming out of IMX412 camera inside your 3D printed enclosure. It should come straight away from the camera not towards to center of the camera PCB.

                              for the configuration file, you could edit it right on VOXL (using a command line text editor, like vi), or you can make a new file voxl-camera-server.conf on your PC with the exact contents that i posted and push it to /etc/modalai/

                              Alex

                              WineVirusW 1 Reply Last reply Reply Quote 0
                              • WineVirusW
                                WineVirus @Alex Kushleyev
                                last edited by

                                @Alex-Kushleyev Is there somewhere I can download the imx412_3.bin file? It's not in the output of ./adb shell ls -la /usr/lib/camera

                                The flex connector comes straight away from the camera.

                                I've pushed the server config file.

                                Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                • Alex KushleyevA
                                  Alex Kushleyev ModalAI Team @WineVirus
                                  last edited by

                                  @WineVirus

                                  copy the IMX412 driver

                                  cp /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_3.bin /usr/lib/camera/

                                  WineVirusW 1 Reply Last reply Reply Quote 0
                                  • WineVirusW
                                    WineVirus @Alex Kushleyev
                                    last edited by

                                    @Alex-Kushleyev ok, the imx412_3 bin file is in /usr/lib/camera/, but I'm still only sensing 1 camera.

                                    Also, I'll be on spring break, so I apologize for a potential lack of response.

                                    Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                    • Alex KushleyevA
                                      Alex Kushleyev ModalAI Team @WineVirus
                                      last edited by

                                      @WineVirus Hmm. please double check to make sure all the connectors are seated properly. They should "snap in" as you insert them.

                                      If that does not work, it could be a bad cable for IMX412 camera, or something else. we can try putting IMX412 on JL, so it would e imx412_2, which you could also copy and try. Then AR0144 could be on JU, so you could download it from here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/master/misc_files/usr/share/modalai/voxl-camera-server/drivers/ar0144/com.qti.sensormodule.ar0144_3.bin

                                      Swapping the cameras can help figure out where the issue is. You can leave the existing sensormodules in there (no need to delete them), just add ar0144_3 and imx412_2

                                      Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                      • Alex KushleyevA
                                        Alex Kushleyev ModalAI Team @Alex Kushleyev
                                        last edited by Alex Kushleyev

                                        If you switch the two cameras around, make sure double check pin 1 on the connector. JL and JU are rotated 180 degrees with respect to each other. You can also just test IMX412 and not connect AR0144 at all, if that is simpler, so that you could test IMX412 only.

                                        WineVirusW 1 Reply Last reply Reply Quote 0
                                        • WineVirusW
                                          WineVirus @Alex Kushleyev
                                          last edited by

                                          @Alex-Kushleyev Ran through the troubleshooting steps (swapping both connections), still only getting one camera (same ID, just on the other slot). I think I'll just replace the cable.

                                          Can I work on the setup on only one sensor, with a simple way to add another one? Or will I have to redo the whole setup when I have both?

                                          voxl2-mini:/$ voxl-camera-server -l
                                          existing instance of voxl-camera-server found, attempting to stop it
                                          DEBUG:   Attempting to open the hal module
                                          DEBUG:   SUCCESS: Camera module opened on attempt 0
                                          DEBUG:   ----------- Number of cameras: 1
                                          
                                          DEBUG:   Cam idx: 0, Cam slot: 3, Slave Address: 0x0030, Sensor Id: 0x0356
                                          DEBUG:   Note: This list comes from the HAL module and may not be indicative
                                          DEBUG:          of configurations that have full pipelines
                                          
                                          DEBUG:   Number of cameras: 1
                                          
                                          VinnyV 1 Reply Last reply Reply Quote 0
                                          • VinnyV
                                            Vinny ModalAI Team @WineVirus
                                            last edited by Vinny

                                            Hi @WineVirus
                                            I think you are also using M0135 wrong.
                                            You can only use 1 of them on each J6 or J7, they cannot be cascaded.
                                            The way I see this here is not the correct usage:
                                            dfc1df9a-4e0d-4cd5-b5ef-0bdda810512b-image.png
                                            So the hires camera will never work like this.
                                            The one camera you do see is the tracking cam... this one:
                                            94deff10-3f1d-44cd-bc54-c14c1d1f5aad-image.png
                                            You must connect the module into M0135 directly only when plugged into the Voxl 2 Mini.
                                            It is not designed to use the extension flex to mate to another image sensor via M0135.
                                            In summary, remove this second M0135 and the M0036 extension flex:
                                            3347e9d8-6d9e-4071-960f-15feba9d5b92-image.png And plug the M0149 directly into M0135 here to mimic what Alex has demonstrated:
                                            80feee73-fcaa-47fd-80ad-77b3c12fe560-image.png
                                            Hope this helps.

                                            If I am mistaken, please let us know but this is what I gather from your photos.

                                            Alex KushleyevA 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Powered by NodeBB | Contributors