Posts made by Alex Kushleyev

@anttukoski , yes I think you are right. I will work with your team via email to provide what you need. I believe we already have a thread.

Alex

@Rowan-Dempster, one reason would be if your video recording resolution is lower than full camera resolution, but you may want the input for ROI to be the maximum resolution (so you get the best zoomed in quality). So in this case you would set your preview resolution to the max (and use preview as input to ROI) and then your video stream would record the downscaled full view.

If your video recording resolution is already at maximum, then there would be no benefit in using a preview stream for this use case.

Depending on the use case, you can choose what works better for you.

Alex

Hi @anttukoski ,

The chi-cdk repository that you already have should contain the source xml files for the sensormodule and tuning files as well as the tools to build them. Additionally you would need a sensor.so (compiled from a cpp source) file for your new camera, which contains exposure and gain conversion functions for the specific camera. It seems you area already familiar with this.

VOXL2 uses the Qualcomm's standard format of camera drivers (from ubun1.0 main release), which you are already familiar, so if you build your sensormodule, tuning file and sensor.so, you should be good.

With this in mind, can you please let me know how we can help you? It seems that you already have everything you need.

Alex

@shawn_ricardo ,

The hardware configuration for your application should be as follows:

• J6 : tracking camera with either M0076 or M0135 interposer. Use J6L (lower camera port), which is the only port available on M0076 or JL port on M0135. This camera will be in HW Slot 0.
• J7 : Hadron, as described in the guide you already found. IR in HW slot 2, OV64B in slot 3
• J8 : hires (any) camera. Use either M0076 or M0135. Use J8L (lower camea port). HW slot 4

• note that J6U and J8U camera slots can only be used by the second stereo camera (in a stereo pair) due to custom I/O configuration that is default for VOXL2 varian 0 (which is what you have on Sentinel).

If you want to swap tracking and hires camera locations, you can just swap all the instructions as well

When connecting interposer and cameras, pay attention to pin1 orientaion, which is labeled on every camera connector, voxl2 connectors and interposers.

In terms of software setup:

• back up (move) all sensormodule files from /usr/lib/camera/
• copy the following sensormodule files that you will find in /usr/share/modalai/chi-cdk (i will assume that you are using imx214 hires camera):

cp /usr/share/modalai/chi-cdk/ov7251/com.qti.sensormodule.ov7251_0.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/boson/com.qti.sensormodule.boson_2.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/ov64b40/com.qti.sensormodule.ov64b40_3.bin /usr/lib/camera/
cp /usr/share/modalai/chi-cdk/imx214/com.qti.sensormodule.imx214_4.bin /usr/lib/camera/

confirm the following are present in /usr/lib/camera/"

com.qti.sensor.ov7251.so
com.qti.sensor.ov64b40.so
com.qti.sensor.boson.so
com.qti.sensor.imx214.so

Then, before updating /etc/modalai/voxl-camera-server.conf you can see if the cameras are detected:

voxl-camera-server -l

See output example here :

• https://docs.modalai.com/debugging-camera-issues/#checking-which-cameras-are-detected-by-camera-server
• https://docs.modalai.com/debugging-camera-issues/#sensor-id-to-sensor-type-mapping

For camera server config, use the current config that you already have, wipe any cameras from it that are not tracking or hires. Then add the two camera entries for Hadron into your config, as shown here : https://docs.modalai.com/voxl2-hadron/#minimum-config .

You will need to make sure the camera IDs specified in voxl-camera-server.conf are correct:
0: ov7251
1: boson
2: ov64b
3: imx214

At this point everything should work, once you get here, I can help you customize the video streams as you need.

Alex

Hi @Rowan-Dempster , I actually tried this branch with both sources for roi (preview and large video stream). It seems to work fine.

I did notice that there is a bug where we were not setting the frame id, this needs to be added right here : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/feature-add-c2d-select-roi-src/src/hal3_camera_mgr.cpp#L1341

roi_meta.frame_id     = meta.frame_id;

So, please try the code in the branch (with frame id fix) with different roi sources and let me know how it behaves. It should not matter whether the camera is IMX214 or IMX412, but i have not tried with IMX214. If this branch has the issue with IMX214 based on your testing, i can try it.

Also, if you have not already done so on your branch, please lock voxl-camera-server to use only upper 4 cores, as it significantly improves camera server performance when the load is high (the low level ISP interrupts happen on lower cores only, so this way camera server is not interfering). The reason why i mention this is that you are using an old branch of camera server, so just double checking.

Alex

Hello @RyanH ,

The GPS interference work is still ongoing. Currently, the solution that works is elevating the GPS using a mast with some shielding right under the mast, as described in the thread you referenced.

We recently added a feature to voxl-portal to allow you to see the individual satellite signal strengths in order to help with debugging the signal quality. The GPS page can be located under the Debug section of voxl-portal, you should get the recent nightly build of voxl-portal in order to try it.

Elevating the GPS receiver by about 4 inches with shielding has shown to completely eliminate the interference, but you will also need an extended cable that plugs between the GPS receiver and VOXL2.

If you need more details about the workaround, I can prepare better documentation and I'll see if we can have replacement cables or extensions available to make the electrical connection easier.

Alex

Hi @IlkkaN ,

The com.qti.tuned.imx412_fpv.bin tuning file was created in attempt to remove some of processing in order to reduce delay and tweak / reduce de-noise / smoothing. I believe the starting point was com.qti.tuned.cmk_imx577.bin which was provided by Qualcomm (you may be able to find that one). IMX577 and IMX412 are nearly identical cameras.

We do not have a tuning file for IMX678 and currently the tuning file name is not specified in the IMX678 sensormodules, which means that com.qti.tuned.default.bin will be used. The default tuning file is the fallback in case the tuning file name specified in the sensormodule file is not found. The default tuning file is not tuned for any specific camera and should be used with caution :).

If you want to confirm which tuning file is used by each camera / sensormodule, you can check this the following way:

add to /vendor/etc/camera/camxoverridesettings.txt:
logInfoMask=0x20000

then run

logcat | grep -i tuning

and then start voxl-camera-server, you should see output like this:

03-30 01:57:45.256  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1762 CreateTuningDataManager() Enter : sensor index: 0, sensor name: cmk_imx577, chromatix name: cmk_imx577
03-30 01:57:45.256  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1798 CreateTuningDataManager() Searching regular tuning file
03-30 01:57:45.262  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1839 CreateTuningDataManager() Loaded tuning file: /usr/lib/camera/com.qti.tuned.cmk_imx577.bin for sensor index: 0
03-30 01:57:45.301  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1762 CreateTuningDataManager() Enter : sensor index: 1, sensor name: ar0144, chromatix name: onsemi_ar0144
03-30 01:57:45.301  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1798 CreateTuningDataManager() Searching regular tuning file
03-30 01:57:45.310  2705  2705 I CamX    : [ INFO][HWL    ] camxhwenvironment.cpp:1839 CreateTuningDataManager() Loaded tuning file: /usr/lib/camera/com.qti.tuned.onsemi_ar0144.bin for sensor index: 1

Our approach is currently not to heavily rely on the ISP tuning, instead we are building our own light-weight image processing pipeline, called MISP, which does processing on the GPU (from RAW bayer frames). MISP also allows us to create up to 6 high resolution color video pipelines, as opposed to the limit of 2 cameras (in video mode) supported by Spectra480 ISP, since it has two separate full VFEs.

By using MISP, we have full control over the algorithms that are running and we can easily share details with the customers. However, MISP definitely lacks many features that are present in the hardware ISP.

Please let me know if you have any other questions.

Alex

@anttukoski , the only way, currently, to connect a third party camera to VOXL2 is using a USB interface, probably using the standard linux UVC driver (voxl-uvc-server would support this)

In order have access to the sources and tools to build the sensormodule camera drivers and tuning files, you would need a partner license from Qualcomm, since this is their terms.

Besides software integration, there is also hardware aspect to supporting the camera, making sure all the connections are properly routed to VOXL2.

Is there a specific camera that you have in mind (and a specific camera module)?

Alex

Hello @JoeAR_21 ,

The debug log shows that the transition from the sinusoidal spin-up to closed-loop (based on back-emf) control is not working. There are several safety checks to prevent a bad motor from spinning up and the checks happen during this transition.

Most likely the motor windings or leads are compromised, either one of the phases is disconnected or there may be a short between two phases.

If you have a milliohm meter, you could accurately verify the phase resistance. A disconnected phase can be verified using a regular multi-meter by checking pairwise resistance between the 3 phases (should be the same and close to zero (in the range of 200-300 mOhm), depending on the specific motor used).

While an ESC issue is also possible, however, it is less likely to occur during a crash, unless the ESC was mechanically damaged.

Alex

@anttukoski , can you please clarify your question :

Is your goal to attach an additional already supported ModalAI camera or to connect a 3rd party camera to Starling 2?

Thank you

Alex

@marian , I was able to replicate it and when running dmesg, I see the following message:

[  390.539854] R0: [VosMCThread][02:38:25.067557]  wlan: [3231:E :PE ] limProcessAssocReqFrame: 914: Max Sta count reached : 2

Changing max_num_sta in hostapd.conf did not help.

After doing a little bit of digging, it seems like this may be a hardware limitation of the wifi hardware on VOXL1 in SoftAP mode : https://android.googlesource.com/kernel/msm.git/+/android-msm-bullhead-3.10-n-preview-1/drivers/staging/qcacld-2.0/CORE/MAC/src/pe/lim/limProcessAssocReqFrame.c#847 .

It could also be a setting somewhere low level and unfortunately we don't have any information on that. Sorry about that.

The wifi module on VOXL1 is QCA6174.

Alex

Hello @Rowan-Dempster ,

I am assuming that you are using non-raw preview stream (YUV) at resolution around 4000x3000 and similar resolution for the large encoded stream. Then the ROI stream is the downscaled version of the preview stream (please correct me if I am wrong).

I just tested this setup using IMX412 camera, and using the following branch (which I believe you used as a starting point):

https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/tree/feature-add-c2d-select-roi-src/

It seems to be running just fine. Please test using the above branch and then if that works, then it's possible that you have some changes that cause an issue. Also try disabling all other cameras and seeing if the issue goes away.

|           Pipe Name |  bytes  | wide |  hgt |exp(ms)| gain | frame id |latency(ms)|  fps |  mbps  | format
|           hires_roi | 1382400 | 1280 |  720 | 33.00 | 4035 |     3752 |     53.3  | 30.0 |  331.8 | NV12
| hires_large_encoded |     723 | 4056 | 3040 | 33.00 | 4035 |     3751 |     80.6  | 30.0 |    0.4 | H265 (P)

If the camera server from the above branch does not work, please provide the following details;

• what camera you are using
• voxl-camera-server.conf config section for that camera (assuming other cameras do not interfere)

Alex

Hello @hgutier ,

Can you please provide output of voxl-camera-server -l, which will scan all the connected cameras? You can just list the top part, something like this (just an example). Please note that this command will stop the background camera server service, if it's running, which can be restarted using systemctl start voxl-camera-server or just rebooting voxl2.

 voxl-camera-server -l
DEBUG:   Attempting to open the hal module
DEBUG:   SUCCESS: Camera module opened on attempt 0
DEBUG:   ----------- Number of cameras: 3

DEBUG:   Cam idx: 0, Cam slot: 0, Slave Address: 0x0020, Sensor Id: 0x0214
DEBUG:   Cam idx: 1, Cam slot: 1, Slave Address: 0x0030, Sensor Id: 0x0356
DEBUG:   Cam idx: 2, Cam slot: 2, Slave Address: 0x0034, Sensor Id: 0x0077

Also, please list the following contents:

ls /usr/lib/camera/*sensormodule*

We will just double check that the software is set up correctly to use the TOF module.

Thank you

Alex

Hi @Morten-Nissov ,

Sorry for the delay. We will get the M0172 board available for purchase ASAP.

Since your configuration uses M0014 camera, which would not be able to plug into M0172 splitter, here is your connection option:

• VOXL2 Mini J6:
  • M0076 or M0135 interposer + M0014 camera
• VOXL2 Mini J7:
  • M0172 interposer + TOF

(You could swap the connections, such as TOF is connected to J6, and vice versa). You would simply need to update the sensormodules (camera drivers) used, according to the slot id of each camera.)

When using VOXL2 mini with TOF, our guidance is to use only one TOF V2 per VOXL2 mini

• the TOF V2 sensor uses 5V power input, however VOXL 2 Mini only has a budget of 900mA on the 5V rail because the input voltage to the mini is 3.8V and there is a step-up regulator to 5.0V. This is different from VOXL2, which uses an APM / ESC to supply 5V 6A into VOXL2.
• the TOF V2 internally uses 3.3V and the pulsed current draw in different operating modes can be found here : https://docs.modalai.com/M0178/#irs2975-operating-modes . Using multiple TOF sensors on voxl2 mini can result in exceeding the current limit on the 5V regulator.

Keep in mind that you will still be able to plug in additional cameras if you need to:

• a uCoax camera plugged into M0172 (M0161, M0166, etc)
• any camera that uses older connector plugged into M0135 (M0014, M0107, etc)

One thing to keep in mind when using M0076 and M0135 is because they were originally designed for VOXL2, though compatible with VOXL2 mini, the interposers may interfere with the mini's mounting holes, because M0076 / M0135 (when inserted) have slight overlap with the mounting holes. However, the overlap may not be an issue if you dont have stand-offs on the side where M0076 / M0135 plugs into VOXL2 mini.

We will update this thread once M0172 is available in the shop.

Alex

@Rowan-Dempster can you please point me to the version of camera server that you are using? Or the base version which you branched off from? Thanks!

Alex

Hi @robertociuch ,

There are no sync issues with AR0144 that we are aware of right now. If you want the cameras to by sync'ed, you have to use the "combo" or "fsin" drivers for the AR0144 camera (see below re fsin). The non-combo / non-fsin drivers do not use the sync line as the sync input and will be free-running.

The only additional limitation for the sync'ed AR0144 vs non-synced, the maximum exposure is limited to 12ms (compared to 33ms in non-synced use case). The actual limitation is slightly higher than 12ms (maybe 14-15) and it is coming from the way the camera operates in the sync'ed mode (this is specific to AR0144 only). This 12ms limit is set in voxl-camera-server (per each camera) "exposure_max_us": 12000, so you want to double check that. If you don't limit the exposure, then if the exposure setting extceeds this limit (well, exceeds 15ms or so), the frame rate will drop in half from 30 to 15. When you run voxl-configure-cameras 26, the correct sensormodules will be coped to /usr/lib/camera/ and correct config with appropriate exposure limits applied to /etc/modalai/voxl-camera-server.conf.

In terms of sensormodule IDs for AR0144, you need to look where they are connected.

If we look at C27 use case, from the following link:
https://docs.modalai.com/voxl2-coax-camera-bundles/

there are 3 AR0144 cameras connected and the following sensormodules are present:

com.qti.sensormodule.ar0144_combo_0.bin
com.qti.sensormodule.ar0144_combo_6.bin
com.qti.sensormodule.ar0144_fsin_2.bin

Actually the combo sensormodules are not only sync'ed but they also use a single MIPI interface to connect two cameras (nice trick). That is really whey they are called combo (the streams are combined into a single mipi interface). The third camera takes up the whole 4-lane mipi camera port and that is why it is not a combo mode driver. fsin means it is using frame sync input.

So all 3 of those camera drivers will use the sync input, generated by VOXL 2 for sync signal. BTW, the voxl-camera-server.conf contains information about the frame sync pin and whether it is enabled or not. You have to make sure that you enable it and use correct pin when using M0173:

	"fsync_en":	true,
	"fsync_gpio":	109,

This is also mentioned here : https://docs.modalai.com/M0173/

If you are only connecting two AR0144 cameras, you should connect them as shown in C26 config and you would want to have the following sensormodules:

com.qti.sensormodule.ar0144_combo_0.bin
com.qti.sensormodule.ar0144_combo_6.bin

Finally, in order to confirm time sync. You can do it indirectly by looking at the timestamp of the frames coming from two cameras. I actually wrote a tool which subscribes to two camera streams and will compare the incoming timestamps to tell you how much out of sync they are. I will find the tool and share it.

There is also a direct way of checking sync of looking at the camera images. We have a special LED array where we can light up each LED for a fixed number of microseconds. Each LED is lit up for that short amount of time and the LEDs light up in a scanning pattern across a 2D array. And then we set the camera exposure to a small value (1ms lets say) for both cameras. In the image below, each LED lights up for 50us and the camera exposure was set to 1ms, so we see 20 LEDs lit up at the same time (50*20 = 1000us) and they are also aligned in time to within 50us). This specific capture was from an older ov7251 test, but we have also done ar0144 sync test.

I just did a quick test using two sync'ed AR144, showing 0 delay in timestamps:

voxl2:~$ ./voxl-cam-sync-check -a tracking_front -b tracking_rear
camera a: /run/mpa/tracking_front/
camera b: /run/mpa/tracking_rear/
ab delta: 0.000000

Of course the sync cannot be EXACTLY zero, but it is as close to zero as the system can measure (way below 100us), just based on the timestamp test.

If I do a comparison with a non-sync'ed hires cameras:

voxl2:~$ ./voxl-cam-sync-check -a tracking_front -b hires_bayer      
camera a: /run/mpa/tracking_front/
camera b: /run/mpa/hires_bayer/
ab delta: 0.053434a ahead, trying to re-sync
ab delta: -0.013322  dt too large!
ab delta: -0.013265  dt too large!
ab delta: -0.013148  dt too large!
ab delta: -0.013226  dt too large!

Please let me know if you have any more questions.

Alex

@robertociuch , thanks for letting me know! I will make sure this update makes it into a future SDK release.

Alex

Hello @SKA ,

I will double check, but i do not think that we have what you are asking for. However, there is M0188 which will allow you to connect up to 4 M0166 tracking cameras and 2 M0161 hires cameras concurrently. Would that work for you?

https://docs.modalai.com/M0188/
https://www.modalai.com/products/m0188

Please note that if you use M0188, you will need to install the VOXL2 mini sdk and select the variant that supports M0188 (it uses a different kernel to map all the camera IO correctly).

Alex

Hi @marian,

Please see the following post regarding VOXL1 wifi AP maximum number of clients : https://forum.modalai.com/topic/3523/maximum-connections-to-wi-fi . Hopefully that helps!

Alex

@h3robotics ,

I am working on merging the current EIS features to our main branch, this should be done sometime next week.

Meanwhile, it would help if you briefly describe your use case for EIS, so I can make sure it is supported in the release. Are you using IMX412 camera?

The Rolling Shutter correction won't yet be supported, but I wanted to release what we have so far, which works pretty under low vibration conditions.

If you don't have the build environment set up for building voxl-camera-server, i can provide the deb package for you to test what is on EIS branch right now.

Alex