@jimbow77 , regarding #4, there are four batteries (two sets).
#3 - we don't have an option to install FLIR Lepton, but you can purchase it separately (from another vendor) and install it. The adapter is present in Starling 2 Max, so you would just need to install the Lepton sensor into the socket.
Alex
@l05 can you try running the ./make_package.sh script inside docker? also if you are just trying to deploy your app, you can copy it directly from voxl-mpa-tools/build64/tools, to voxl2 using adb or ssh (assuming the build itself succeeded).
Alex
here is what the TOF sensor mounting looks like, right next to front-facing IMX412 and AR0144 cameras.
Starling 2 and Starling 2 Max are very similar in terms of what software they are running. If you are able to run the mapping / navigation on a Starling 2, then you should be able to follow the same steps to get it to work on Starling 2 Max (sorry for lack of details, but it seems you just want to confirm that the same functionality is supported, and the answer is yes). Please note that getting the navigation and mapping to work well in your application will require testing and tuning.
I will check regarding 3 and 4 and get back to you.
Alex
@l05 , can you please clarify the following: do you need to receive images from camera server in a python script or a C/C++ application?
The recommended way is to use a C/C++ application, for which we can provide some examples. Python extension via Pympa makes it easier to quickly test / prototype something, but may have additional overhead to receive / convert images.
For example, there is a tool voxl-inspect-cam-ascii, which is installed by default with the VOXL2 SDK, which subscribes to an image and renders the image to your terminal using ASCII. It uses opencv to downscale the image. You can find the source code here : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-mpa-tools/-/blob/master/tools/voxl-inspect-cam-ascii.cpp . this app also shows how to publish an image (it optionally publishes the down-scaled image as well).
You can test this app by running voxl-inspect-cam-ascii <camera_stream> in a terminal, and you should see an ASCII version of your image. The name of the camera stream you can get from voxl-portal (camera tab), for example it would be something like tracking or hires_front_color, depending on the cameras / streams that you have enabled.
If you wanted to develop your own application, you can start by first building the voxl-mpa-tools package from source and then add your own application into the existing build structure.
Alex
@chengyouzen , have you tried building OpenSSL from source (you should be able to do it directly on VOXL1)? That may be the only way..
Alex
@SKA ,
In your case, the issue should be solved by commenting out one line in https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/eis/src/misp.cpp :
//TODO: FIXME
//in roll following mode, the roll is not stabilized to the horizon, so it does not need to be flipped
//need to resolve this in a better way because Rout is meant for another purpose
if (!follow_roll)
rc_matrix_right_multiply_inplace(&eis_ctx.H.m, eis_ctx.Rout.m); //rotate the image according to desired output rotation (nominally identity)
Specifically, comment out the if statement : if (!follow_roll) , so that the Rout matrix is always applied to the full transform H.
The issue is that I tried to solve another use case by using this parameter (when the voxl2 / IMU is flipped upside down while camera is right side up). I just tested and it appears to be working.
I will figure out a better way of handling the upside down IMU in this case so that the output transform can be used correctly, but for now you can just comment out that if statement.
Can you please let me know if that fix worked for you?
Alex
@SKA , OK, so what you are saying is that in full-follow mode, the rotation parameter in the extrinsics hires -> hires_eis for the -90 degree rotation is not respected, correct?
@SKA , I thought i fixed that issue a while ago, let me double check. Are you using voxl-camera-server from eis branch or an official release from an SDK?
Alex
@l05 , please see the following post, you can download the voxl-opencv library with python3 bindings from here .
https://forum.modalai.com/topic/3519/help-with-pip-installing-opencv
This version of opencv was used for testing the pympa-experimental tools.
The script you were looking at is setup to receive image frames from an rtsp stream - is that what your goal is? (as opposed to receiving images from a mipi camera connected to VOXL2 (via voxl-camera-server)).
Alex
@dvz ,
The lens in the MSU-M0149 camera module should come with IR filter installed, which means that you will not be able to detect IR light.
You can test with an IR LED or a phone that has an IR scanner (many phones do) or explicitly check for the filter by unscrewing the lens.
The IR filters installed in the lenses inside MSU-M0149 are not designed to be easily removable. Attempting to do so will void the warranty and may cause damage to the camera / lens or personal injury. Small pieces of the glass filter could come loose. Do not do it.
I just tried to remove the IR filter and i was able to remove it, the filter was broken in process. I unscrewed the lens from the lens holder. The adhesive holding the lens in lens holder, if present, is not permanent and can be scraped off if needed. Then I used a sharp point of an x-acto knife to pry the IR lens off. See images below.
We do not have an option to purchase M0149 camera without an IR filter.
However, a newer version of this camera (https://www.modalai.com/products/msu-m0166) has an option to select a lens without IR filter. Please note that M0166 camera uses different (ucoax) cables and adapters for connecting to VOXL2, so this is something to consider if you do decide to switch.
If you would like to purchase a lens without IR filter (it is very similar to the one in MSU-M0149, but without IR filter, similar overall specs), please send us a request, the part number is JSD5501-C0 (mention no IR filter, just in case). This is the lens shipped with M0166 with no IR filter option. https://www.modalai.com/pages/contact-us
Warning : Attempting to remove the IR filter will void the warranty and may cause damage to the camera / lens or personal injury
save image on voxl:
voxl-record-raw-image boson640_bayer -n 1 -d ./
display image using python3:
import cv2
import numpy as np
WIDTH = 640
HEIGHT = 512
FILENAME = "boson640_bayer_640x512.gray"
# read 16-bit file
img = np.fromfile(FILENAME, dtype=np.uint16).reshape(HEIGHT, WIDTH)
# simple normalize
vmin = np.min(img)
vmax = np.max(img)
img_8bit = cv2.convertScaleAbs(img-vmin, alpha=255.0/(vmax-vmin))
# show image
cv2.imshow("Normalized Image", img_8bit)
cv2.waitKey(3000)
cv2.destroyAllWindows()
@mkriesel , it looks like potentially an little vs big endian issue when using PGM - can you try to swap the byte order?
Alex
@mkriesel , I am glad that you got it working! You should also be able to view the boson output in voxl-portal -- for the 14 bit case, there is a very simple re-scaling done from 14 to 8 bit, just for testing (WIP).
I am using SDK4.0_Boson_Plus_SDK. You should probably contact Flir to get access to it, since we can't distribute it.
Alex
Hello @LV-Office-Holder,
The M0138 racing ESC only has two I/O pins and they are used for aux PWM output. PWM input is disabled for this ESC due to lack of PWM input for each ESC channel.
The ESC protocol example can be found here : https://gitlab.com/voxl-public/support/esc-driver as well as part of PX4 implementation : https://github.com/modalai/px4-firmware/tree/voxl-dev/src/drivers/actuators/voxl_esc . I would suggest referencing the PX4 implementation since it has the latest updates.
Additionally, we have voxl-esc test tools (python) : https://gitlab.com/voxl-public/voxl-sdk/utilities/voxl-esc . The test tools allow you to spin motors, plot data, perform firmware and parameter updates.
Please let me know if you have any other questions.
Alex
@jameskuesel , it looks like EIS was merged to the main branch shortly after SDK 1.5.0 was released.
If you have not updated you camera server, please do so : https://docs.modalai.com/camera-video/electronic-image-stabilization/#latest-source-code . You may also need to update libmodal-pipe library.
Alex
@mkriesel , I have updated the boson configuration and read scripts (boson_read.py, boson_set_8bit.py, etc) to include configuring additional settings like mipi start state, dvo output interface, etc. Please download them again (from links earlier in this thread) and try again. Also, just to confirm that the original Boson setup instruction using the FLIR GUI should work for 8 bit format.
Just one note, even though I set myCam.dvoSetOutputFormat(FLR_DVO_OUTPUT_FORMAT_E.FLR_DVO_IR16), where FLR_DVO_IR16 = 3, when i read back the configuration, i get the default format (2) : FLR_DVO_OUTPUT_FORMAT_E.FLR_DVO_DEFAULT_FORMAT = 2 . It seems that in your output the ouput format is set to 3 - i am not sure why, there could be some minor quirks in the boson firmware.
In any case, please try the GUI and updated scripts and let me know how it goes!
Alex
Please make sure that you only have one sensormodule file for boson for the given configuration in /usr/lib/camera/.
One thing to check is to see if the Boson is in active mipi state by default. You can add this to your boson_read.py file :
print('dvoGetMipiStartState: ')
result, mipi_start_state = myCam.dvoGetMipiStartState()
print(result)
print(mipi_start_state)
print('dvoGetMipiState: ')
result, mipi_state = myCam.dvoGetMipiState()
print(result)
print(mipi_state)
My output is:
dvoGetMipiStartState:
FLR_RESULT.R_SUCCESS
FLR_DVO_MIPI_STATE_E.FLR_DVO_MIPI_STATE_ACTIVE
dvoGetMipiState:
FLR_RESULT.R_SUCCESS
FLR_DVO_MIPI_STATE_E.FLR_DVO_MIPI_STATE_ACTIVE
FLR_DVO_MIPI_STATE_ACTIVE is 2, FLR_DVO_MIPI_STATE_OFF is 0
If your sensor does not show active start state, then it may need to be set using the following:
myCam.dvoSetMipiStartState(FLR_DVO_MIPI_STATE_E.FLR_DVO_MIPI_STATE_ACTIVE)
If this still does not work, it is possible that my script does not do the full setup for mipi output, so you may want to try using the FLIR GUI for the initial setup (for 8 bit post AGC and use the default 8 bit sensormodule): https://docs.modalai.com/M0153/#boson-software-setup
Alex
I believe you need to update your voxl-mongoose library to latest, from here : http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64/ . I believe voxl-portal recently switched to using updated version of that package and that is what is being pulled in when you build voxl-portal from dev or EIS branch.
Alex
Even if the checkerboard is detected, it needs to fill the majority of the non-shaded region. In order to progress to the next step, you need to bring the camera / pattern closer together and then the procedure will accept it. I know this is not obvious when you do it for the first time.
Regarding voxl-camera-server freeze, what if you run voxl-inspect-cam <camera_stream> on voxl2, does it still show images coming in?
Try setting your voxl into perf mode (voxl-set-cpu-mode perf) which will speed up the calibration pattern detection. I see that the calibration is only running at 2.8FPS at 1280x800 resolution.
During calibraiton, you may also want to stop other services that might be doing image processing, so that VOLX2 has more cpu cycles for the calibration app.
Alex
Hi @Riccardo-Benedetti and @paul-foley ,
This issue is not common. There is a potential condition (which we are going to look into) when a MPA client crashes and does not close the pipes properly, the MPA server side will keep the allocated resources open. If the client crashes repeatedly, eventually there will be no more memory for the Kernel to allocate for the new pipes.
Please check to make sure that you have no client processes that are subscribing to image streams and are crashing and restarting (perhaps restarting automatically by systemd). It is possible that a process that is not your test app is misbehaving for some reason and is causing this memory leak by continuous crashing and re-starting.
Also, what if you just use voxl-inspect-cam to inspect the single camera stream and no other camera clients running, does the same issue happen?
Alex