• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Register
  • Login
ModalAI Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
    • Register
    • Login

    VOXL 2 M0135 Dual Camera Connections with M0107 and M0149 Cameras

    Ask your questions right here!
    3
    30
    1.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      Alex Kushleyev ModalAI Team @Alex Kushleyev
      last edited by 28 Mar 2024, 01:22

      Hi, I just connected the two cameras to voxl-2 mini running SDK 1.1.1.

      For ar0144, please use the latest sensormodule file from this location . The one that comes with SDK 1.1.1 has stability issues.

      #from your PC
      adb push ~/Downloads/com.qti.sensormodule.ar0144_2.bin /usr/lib/camera/
      
      #now on VOXL2 mini
      voxl2-mini:/$ cp /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_3.bin /usr/lib/camera/
      
      voxl2-mini:/$ voxl-camera-server -l
      DEBUG:   Attempting to open the hal module
      DEBUG:   SUCCESS: Camera module opened on attempt 0
      DEBUG:   ----------- Number of cameras: 2
      
      DEBUG:   Cam idx: 0, Cam slot: 2, Slave Address: 0x0030, Sensor Id: 0x0356
      DEBUG:   Cam idx: 1, Cam slot: 3, Slave Address: 0x0034, Sensor Id: 0x0577
      DEBUG:   Note: This list comes from the HAL module and may not be indicative
      DEBUG:   	of configurations that have full pipelines
      
      DEBUG:   Number of cameras: 2
      ...
      

      The /etc/modalai/voxl-camera-server.conf should look like this (a good starting point):

      {
      	"version":	0.1,
      	"cameras":	[{
      			"type":	"ar0144",
      			"name":	"tracking",
      			"enabled":	true,
      			"camera_id":	0,
      			"fps":	30,
      			"en_rotate":	false,
      			"en_preview":	true,
      			"preview_width":	1280,
      			"preview_height":	800,
      			"en_raw_preview":	true,
      			"ae_mode":	"lme_msv",
      			"ae_desired_msv":	60,
      			"exposure_min_us":	20,
      			"exposure_max_us":	33000,
      			"gain_min":	54,
      			"gain_max":	8000,
      			"exposure_soft_min_us":	5000,
      			"ae_filter_alpha":	0.600000023841858,
      			"ae_ignore_fraction":	0.20000000298023224,
      			"ae_slope":	0.05000000074505806,
      			"ae_exposure_period":	1,
      			"ae_gain_period":	1
      		}, {
      			"type":	"imx412",
      			"name":	"hires",
      			"enabled":	true,
      			"camera_id":	1,
      			"fps":	30,
      			"en_preview":	false,
      			"preview_width":	640,
      			"preview_height":	480,
      			"en_raw_preview":	false,
      			"en_small_video":	true,
      			"small_video_width":	1280,
      			"small_video_height":	800,
      			"small_venc_mode":	"h265",
      			"small_venc_br_ctrl":	"cqp",
      			"small_venc_Qfixed":	30,
      			"small_venc_Qmin":	15,
      			"small_venc_Qmax":	40,
      			"small_venc_nPframes":	9,
      			"small_venc_mbps":	2,
      			"en_large_video":	true,
      			"large_video_width":	3840,
      			"large_video_height":	2160,
      			"large_venc_mode":	"h265",
      			"large_venc_br_ctrl":	"cqp",
      			"large_venc_Qfixed":	38,
      			"large_venc_Qmin":	15,
      			"large_venc_Qmax":	50,
      			"large_venc_nPframes":	29,
      			"large_venc_mbps":	30,
      			"en_snapshot":	false,
      			"en_snapshot_width":	4056,
      			"en_snapshot_height":	3040,
      			"ae_mode":	"isp"
      		}]
      }
      

      then you can start voxl-camera-server in regular mode (without -l), it should run.

      use voxl-inspect-cam -a to check video streams:

      |           Pipe Name |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  | format
      |   hires_large_color |12441600 | 3840 | 2160 | 33.23 | 1125 |      566 |     78.3  | 29.9 | 2974.3 | NV12
      | hires_large_encoded |    4893 | 3840 | 2160 | 33.23 | 1125 |      566 |     89.2  | 29.9 |    1.3 | H265 (P)   
      |    hires_large_grey | 8294400 | 3840 | 2160 | 33.23 | 1125 |      567 |     70.6  | 29.9 | 1982.9 | RAW8
      |   hires_small_color | 1536000 | 1280 |  800 | 33.23 | 1125 |      568 |     57.3  | 29.9 |  367.2 | NV12
      | hires_small_encoded |    2710 | 1280 |  800 | 33.23 | 1125 |      568 |     60.9  | 29.9 |    1.8 | H265 (P)   
      |    hires_small_grey | 1024000 | 1280 |  800 | 33.23 | 1125 |      568 |     55.5  | 29.9 |  244.8 | RAW8
      |            tracking | 1024000 | 1280 |  800 |  7.04 |  665 |     8043 |     20.7  | 30.0 |  245.6 | RAW8
      

      Then you can use voxl-portal to view images (assuming you have a wired or wifi adapter plugged into your voxl2-mini:
      (if voxl portal is not enabled, you can enable it):

      systemctl enable voxl-portal
      systemctl start voxl-portal
      

      Then go to web browser and enter your voxl2-mini's ip to view voxl-portal

      A W 3 Replies Last reply 28 Mar 2024, 01:26 Reply Quote 0
      • A
        Alex Kushleyev ModalAI Team @Alex Kushleyev
        last edited by 28 Mar 2024, 01:26

        images of my setup attached

        IMG_3880.jpg
        IMG_3881.jpg

        1 Reply Last reply Reply Quote 0
        • W
          WineVirus @Alex Kushleyev
          last edited by WineVirus 28 Mar 2024, 22:38 28 Mar 2024, 21:54

          @Alex-Kushleyev Here's what I'm seeing: bc33809b-869c-4beb-a235-a691d3cde4be-image.png

          Here's the wiring; It looks the same as yours. IMG_1311 (1).jpg IMG_1312 (1).jpg

          A 1 Reply Last reply 28 Mar 2024, 22:14 Reply Quote 0
          • A
            Alex Kushleyev ModalAI Team @WineVirus
            last edited by 28 Mar 2024, 22:14

            @WineVirus i believe you have M0135 plugged into J6 instead of J7 like I do. Take a look at my photo again.

            W 1 Reply Last reply 29 Mar 2024, 21:41 Reply Quote 0
            • W
              WineVirus @Alex Kushleyev
              last edited by 29 Mar 2024, 21:41

              @Alex-Kushleyev Ahh, yeah. My board is upside down, so I got turned around.

              I'm now detecting one camera, but both are plugged in. I'm not sure which one this is?

              voxl2-mini:/$ voxl-camera-server -l
              DEBUG:   Attempting to open the hal module
              DEBUG:   SUCCESS: Camera module opened on attempt 0
              DEBUG:   ----------- Number of cameras: 1
              
              DEBUG:   Cam idx: 0, Cam slot: 2, Slave Address: 0x0030, Sensor Id: 0x0356
              DEBUG:   Note: This list comes from the HAL module and may not be indicative
              DEBUG:          of configurations that have full pipelines
              
              DEBUG:   Number of cameras: 1  
              

              Also, in terms of updating the config - is it a push command, like the ar0144 driver? Not sure what command to use in ADB.

              A 1 Reply Last reply 29 Mar 2024, 21:52 Reply Quote 0
              • A
                Alex Kushleyev ModalAI Team @WineVirus
                last edited by 29 Mar 2024, 21:52

                @WineVirus please double check that you have com.qti.sensormodule.imx412_3.bin in /usr/lib/camera/ and also double check the orientation of the flex cable coming out of IMX412 camera inside your 3D printed enclosure. It should come straight away from the camera not towards to center of the camera PCB.

                for the configuration file, you could edit it right on VOXL (using a command line text editor, like vi), or you can make a new file voxl-camera-server.conf on your PC with the exact contents that i posted and push it to /etc/modalai/

                Alex

                W 1 Reply Last reply 29 Mar 2024, 22:07 Reply Quote 0
                • W
                  WineVirus @Alex Kushleyev
                  last edited by 29 Mar 2024, 22:07

                  @Alex-Kushleyev Is there somewhere I can download the imx412_3.bin file? It's not in the output of ./adb shell ls -la /usr/lib/camera

                  The flex connector comes straight away from the camera.

                  I've pushed the server config file.

                  A 1 Reply Last reply 29 Mar 2024, 22:23 Reply Quote 0
                  • A
                    Alex Kushleyev ModalAI Team @WineVirus
                    last edited by 29 Mar 2024, 22:23

                    @WineVirus

                    copy the IMX412 driver

                    cp /usr/share/modalai/chi-cdk/imx412/com.qti.sensormodule.imx412_3.bin /usr/lib/camera/

                    W 1 Reply Last reply 29 Mar 2024, 22:29 Reply Quote 0
                    • W
                      WineVirus @Alex Kushleyev
                      last edited by 29 Mar 2024, 22:29

                      @Alex-Kushleyev ok, the imx412_3 bin file is in /usr/lib/camera/, but I'm still only sensing 1 camera.

                      Also, I'll be on spring break, so I apologize for a potential lack of response.

                      A 1 Reply Last reply 29 Mar 2024, 22:46 Reply Quote 0
                      • A
                        Alex Kushleyev ModalAI Team @WineVirus
                        last edited by 29 Mar 2024, 22:46

                        @WineVirus Hmm. please double check to make sure all the connectors are seated properly. They should "snap in" as you insert them.

                        If that does not work, it could be a bad cable for IMX412 camera, or something else. we can try putting IMX412 on JL, so it would e imx412_2, which you could also copy and try. Then AR0144 could be on JU, so you could download it from here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/master/misc_files/usr/share/modalai/voxl-camera-server/drivers/ar0144/com.qti.sensormodule.ar0144_3.bin

                        Swapping the cameras can help figure out where the issue is. You can leave the existing sensormodules in there (no need to delete them), just add ar0144_3 and imx412_2

                        A 1 Reply Last reply 29 Mar 2024, 22:53 Reply Quote 0
                        • A
                          Alex Kushleyev ModalAI Team @Alex Kushleyev
                          last edited by Alex Kushleyev 29 Mar 2024, 22:53 29 Mar 2024, 22:53

                          If you switch the two cameras around, make sure double check pin 1 on the connector. JL and JU are rotated 180 degrees with respect to each other. You can also just test IMX412 and not connect AR0144 at all, if that is simpler, so that you could test IMX412 only.

                          W 1 Reply Last reply 8 Apr 2024, 21:57 Reply Quote 0
                          • W
                            WineVirus @Alex Kushleyev
                            last edited by 8 Apr 2024, 21:57

                            @Alex-Kushleyev Ran through the troubleshooting steps (swapping both connections), still only getting one camera (same ID, just on the other slot). I think I'll just replace the cable.

                            Can I work on the setup on only one sensor, with a simple way to add another one? Or will I have to redo the whole setup when I have both?

                            voxl2-mini:/$ voxl-camera-server -l
                            existing instance of voxl-camera-server found, attempting to stop it
                            DEBUG:   Attempting to open the hal module
                            DEBUG:   SUCCESS: Camera module opened on attempt 0
                            DEBUG:   ----------- Number of cameras: 1
                            
                            DEBUG:   Cam idx: 0, Cam slot: 3, Slave Address: 0x0030, Sensor Id: 0x0356
                            DEBUG:   Note: This list comes from the HAL module and may not be indicative
                            DEBUG:          of configurations that have full pipelines
                            
                            DEBUG:   Number of cameras: 1
                            
                            VinnyV 1 Reply Last reply 9 Apr 2024, 05:36 Reply Quote 0
                            • VinnyV
                              Vinny ModalAI Team @WineVirus
                              last edited by Vinny 9 Apr 2024, 05:39 9 Apr 2024, 05:36

                              Hi @WineVirus
                              I think you are also using M0135 wrong.
                              You can only use 1 of them on each J6 or J7, they cannot be cascaded.
                              The way I see this here is not the correct usage:
                              dfc1df9a-4e0d-4cd5-b5ef-0bdda810512b-image.png
                              So the hires camera will never work like this.
                              The one camera you do see is the tracking cam... this one:
                              94deff10-3f1d-44cd-bc54-c14c1d1f5aad-image.png
                              You must connect the module into M0135 directly only when plugged into the Voxl 2 Mini.
                              It is not designed to use the extension flex to mate to another image sensor via M0135.
                              In summary, remove this second M0135 and the M0036 extension flex:
                              3347e9d8-6d9e-4071-960f-15feba9d5b92-image.png And plug the M0149 directly into M0135 here to mimic what Alex has demonstrated:
                              80feee73-fcaa-47fd-80ad-77b3c12fe560-image.png
                              Hope this helps.

                              If I am mistaken, please let us know but this is what I gather from your photos.

                              A 1 Reply Last reply 9 Apr 2024, 13:11 Reply Quote 0
                              • A
                                Alex Kushleyev ModalAI Team @Vinny
                                last edited by Alex Kushleyev 9 Apr 2024, 13:11 9 Apr 2024, 13:11

                                @Vinny , i was able to use both cameras in this configuration (see above in this thread).

                                It seems the imx412 camera is not detected, potentially bad extension flex cable or (less likely) bad camera. Has this hires camera worked before at some point?

                                VinnyV 1 Reply Last reply 9 Apr 2024, 16:05 Reply Quote 0
                                • VinnyV
                                  Vinny ModalAI Team @Alex Kushleyev
                                  last edited by 9 Apr 2024, 16:05

                                  @Alex-Kushleyev OK, I am just making sure the M0135 is not used in a cascaded mode (a M0135 plugged into another M0135). That is what the first pictures look like to me since the M0135 is just sitting open on a bench.

                                  As far as the image sensor, all modules are tested at ModalAI HQ before shipping in a known good working configuration of hardware (i.e.: a golden setup). So, yes, all image sensors are working when shipped. Same for the extension flexes. Every one is tested before shipping.

                                  I do see a bend in the flex right here, so hopefully that is not a firm crease since creases will damage flexes: e3cd096d-78c1-4934-b59b-169d66df18bf-image.png
                                  Hope this helps!

                                  W 1 Reply Last reply 9 Apr 2024, 21:33 Reply Quote 0
                                  • W
                                    WineVirus @Vinny
                                    last edited by 9 Apr 2024, 21:33

                                    @Vinny I think you may be mistaken... One end of the M0036 is connected to the M0135, and the other end is connected to the M0107 Camera (that's inside the purple case). We're only using one M0135.

                                    I'm receiving a new M0036 cable tomorrow. I'll test that.

                                    VinnyV 1 Reply Last reply 9 Apr 2024, 21:43 Reply Quote 0
                                    • VinnyV
                                      Vinny ModalAI Team @WineVirus
                                      last edited by 9 Apr 2024, 21:43

                                      @WineVirus Thanks for confirming!
                                      Just trying to help and that picture struck me as odd.

                                      Glad you have it covered with Alex, he'll guide you correctly!!!

                                      1 Reply Last reply Reply Quote 0
                                      • W
                                        WineVirus @Alex Kushleyev
                                        last edited by WineVirus 9 Apr 2024, 22:05 9 Apr 2024, 22:02

                                        @Alex-Kushleyev I'm trying to run voxl-camera-server and I'm getting error messages for both cameras; The hires camera makes sense, but I'm not sure about the tracking cam...

                                        I've swapped the M0149 back to the JL port where it was originally. Also, I personally haven't gotten the hires camera working, but I agree that it's more likely the cable.

                                        Starting Camera: tracking
                                        Using gain limits min: 54 max: 863
                                        ERROR:   Camera 0 failed to find supported preview config: 640x480
                                        ERROR:   Failed to start camera: tracking
                                        Starting Camera: hires
                                        ERROR:   failed to call HAL3 get_camera_info for cam 1
                                        ERROR:   Failed to start camera: hires
                                        
                                        ------ voxl-camera-server: Camera server is now running 
                                        

                                        I'm not getting any output. Also, voxl-inspect-services lists voxl-camera-server | Enabled | Not Running | (I'm assuming it's because it has no cameras to run). Voxl Portal and voxl-inspect-cam show no camera streams.

                                        image_2024-04-09_150047903.png

                                        A 1 Reply Last reply 9 Apr 2024, 22:43 Reply Quote 0
                                        • A
                                          Alex Kushleyev ModalAI Team @WineVirus
                                          last edited by Alex Kushleyev 9 Apr 2024, 22:44 9 Apr 2024, 22:43

                                          @WineVirus , i would suggest retracing the steps you took to get the tracking camera to work (from the beginning, after i posted the directions and pictures).

                                          Once you get the tracking camera working, you can try to test the flex by connecting the tracking camera using the flex cable (please wat ch carefully the pin1 orientations). If the flex cable works for tracking camera, it is possible that imx412 has an issue.

                                          I will be back in office tomorrow and can provide more guidance for testing.

                                          A 1 Reply Last reply 10 Apr 2024, 20:13 Reply Quote 0
                                          • A
                                            Alex Kushleyev ModalAI Team @Alex Kushleyev
                                            last edited by Alex Kushleyev 10 Apr 2024, 20:14 10 Apr 2024, 20:13

                                            Actually, i believe in your test, the tracking camera was detected, but you requested unsupported resolution of 640x480. AR0144 only supports raw resolution of 1280x800.

                                            You can see that the error for tracking camera is ERROR: Camera 0 failed to find supported preview config: 640x480 as opposed to hires camera ERROR: failed to call HAL3 get_camera_info for cam 1

                                            So, you should check your voxl-camera-server.conf to make sure that the preview resolution is set to 1280x800 for the AR0144 tracking camera, then it should work.

                                            After that, in order to check if the Flex cable is OK, you can insert the flex cable into the connection between M0135 and AR0144 camera, double check the pin1 orientations.

                                            W 1 Reply Last reply 10 Apr 2024, 22:14 Reply Quote 0
                                            • First post
                                              Last post
                                            Powered by NodeBB | Contributors