Starling 2 Thermal Camera Suggestions
-
Hi devs,
I recently received the Starling 2 drone for our lab and went trough the bootcamp.
For our use case we will need to add a thermal camera, to be able to stream/save the thermal video.Based on the documentation, I found that the FLIR Lepton as well as the FLIR Boson seem to be the most straight forward possibilities.
Nevertheless, I'm not sure about the details on how to add the cameras to the drone.FLIR Lepton:
Are there any updates about the FLIR Lepton camera integration?
I'm aware that the Starling already comes with an added Lepton connector, but I haven't read anything about it being already running and integrated with the software.In case a I/O Module is required, can you suggest which of the available boards to chose from?
Mini USB: https://groupgets.com/products/purethermal-mini-usb
JST: https://groupgets.com/products/purethermal-mini-pro-jst-sr
Other: https://groupgets.com/products/purethermal-breakout-boardAccording to the online guide (https://docs.modalai.com/voxl-uvc-flir/) for VOXL 1 (not sure if this is also valid for VOXL2?), at least an additional USB expansion board will be required.
Is this also true for using the Lepton with the MINI USB I/O board or could I actually connect it via the VOXL2 flightcomputer's USB C interface?
In case this is not an option, do the other I/O board require additional hardware as well or is there a more straight forward option for the VOXL2?FLIR Boson
Regarding the FLIR BOSON option I have found this documentation:
https://docs.modalai.com/m0153/
Can we already purchase this adapter to enable a straight forward connection?Alternatively, I found the following FLIR Boson I/O board online: https://www.flircameras.com/rhp-bos-usbc-if.htm
Would it be an option to connect the board with the Starling flightcomputer's USB-C?Thank you very much for your help!
-
Hi @wifa799 ,
For FLIR Lepton.
This one would work off the M0173 J6 connector:
https://groupgets.com/products/purethermal-breakout-boardI need to update the M0173 data sheet... but a here's pinout for that connector for now for reference, for Lepton 3.5 BTW:
For FLIR Boson.
MIPI Boson:
The M0153-1 is intended for a MIPI based Boson. Currently, the only way to get BOSON into MIPI mode (as shown on the link you provided), is to use a USB backpack (M0153-2) first to put the sensor into MIPI mode (along with other settings). Also, when buying a Boson with MIPI support, it needs to be a Boson+. You will receive it and need to use M0153-2 over USB to connect to it to a Windows machine and then setup for MIPI which can then be used with VOXL using MIPI camera ports, and use VOXL Camera ServerUSBC Boson:
The VOXL2 hardware needs to have M0154 on the silkscreen (not M0054) for the USBC port to work as a host. I just validated a USBC backback on a Boson to the USBC port on VOXL2.Using the USB2.0 port (e.g. where the WiFi dongle is plugged in) would work for the USB2.0 backback. VOXL UVC server would be used here for the USB cameras.
Please keep hitting me with questions here if not clear!
-
@modaltb Thanks a lot for your help!
I have now tried to run the FLIR Lepton connected to this board (https://media.digikey.com/pdf/Data Sheets/GroupGets PDFs/PURETHERMAL-M_3-14-19.pdf) to limit the amount of additional hardware changes. For this I connected a USB splitter to the USB port, where the WiFi dongle is. Then I added the Flir Lepton via USB as well as the WiFi dongle (see photo below).
By typing in "lsusb" I can see the camera as a connected webcam device.
Then I configure the voxl-uvc-server with "voxl-uvc-server -r 160x120 -f 9" and the voxl-streamer with "voxl-streamer -c uvc-flir-lepton".When I switch to the VOXL web portal, I can actually see that an additional "UVC" camera stream is available.
However, it doesn't show the video stream (see attached screenshot) and shows a resolution of "0x0" and a format type of "IMAGE_FORMATt_YUV422_UYVY".
I was also unable to start the stream using VLC as described in the manual (https://docs.modalai.com/voxl-uvc-flir/).
But I was able to see the video as an ASCII stream, so it seems like some data is actually being streamed.
Is it possible that there is a problem with the video format being used?I have tested the camera and breakout board on my laptop and it works fine.
I have also tried accessing the video via QGroundcontrol but have had no luck.
Do you know of any way to actually switch between the different video streams in QGroundcontrol?
At the moment I can only open the high-resolution front camera stream.Thanks again for your help!
-
@modaltb I just wanted to follow up on the issue I described before.
Do you know what causes this behavior?Thanks for your help!
-
@wifa799 What is the output you see when running
voxl-uvc-server -r 160x120 -f 9on the command line? And then if you stop voxl-streamer (e.g.systemctl stop voxl-streamer) and start on the command line withvoxl-streamer -c uvc-flir-leptonwhat does it print out? -
@wifa799 If you have setup and configured multiple RTSP video streams on different ports with voxl-streamer then you can setup the voxl-mavcam-manager application to advertise the streams to QGC. Then QGC will allow you to choose which stream you want. Without voxl-mavcam-manager you can only configure QGC to look at one stream.
-
@Eric-Katzfey Thanks for your reply!
When running
voxl-uvc-server -r 160x120 -f 9I get the following:
So it looks like the voxl-uvc-server is already running in the background.
When I open VLC Network Stream (udp://@:4242) I can't see the camera stream.After I stopped it with CTRL-C, I typed
systemctl stop voxl-streamerand startedvoxl-streamer -c uvc-flir-lepton.
This is the output:
When checking the stream via VLC I still don't see anything.
-
@wifa799 So I would stop both voxl-uvc-server (
systemctl stop voxl-uvc-server) and voxl-streamer (systemctl stop voxl-streamer). Then in one terminal window start voxl-uvc-server from the command line (voxl-uvc-server -r 160x120 -f 9) and leave it running (Don't hit ctrl-c to exit). Then in another terminal first make sure that/etc/modalai/voxl-streamer.confhas been configured properly to use the uvc stream and then start it with verbose output (voxl-streamer -v 0). You can get some documentation on the options withvoxl-streamer --help. Then you should be able to access the RTSP stream with VLC or QGC atrtsp://127.0.0.1:8900/live -
@Eric-Katzfey and @modaltb Thanks for your help!
Unfortunately, I still couldn't get the camera to work.
To make sure it wasn't an SDK issue, I updated it to voxl-suite 1.4.1 and ran the drone's wifi in station mode.
Both the drone and the Linux laptop are connected to my smartphone's hotspot.As suggested, I first stopped the voxl-uvc-server and the voxl-streamer.
Then I started the voxl-uvc-server with the correct Lepton camera configuration and got this error:"uvc_get_stream_ctrl_format_size failed
width, height, and or framerate likely not supported
run voxl-uvc-server -l to list supported resolutions for your camera"
To do what you suggested, I opened a new terminal window and opened the /etc/modalai/voxl-streamer.conf file.
I then started the streamer with verbose output and saw that, as expected, the configuration had not been updated:
To check if there was an actual stream available, I ran nmap on the laptop and saw that port 8900 was still not open:
As expected I also didn't see any RTSP stream using VLC.
Is there anything else I can try?
-
@wifa799 Did you see the error message that was printed when voxl-uvc-server exited? It says to try
voxl-uvc-server -lto see what formats your camera advertises since the one you tried isn't supported. Can you try that? And voxl-streamer configuration file specifies the small encoded hires stream which it appears to connect with and start streaming. You would need to modify the configuration file if you want it to use a different stream. But if you want it to use the voxl-uvc-stream then you need to get that working before you can try streaming it. -
@Eric-Katzfey Thanks again!
First, we stop any running uvc-server and streamer. Then we define the voxl-uvc-server settings usingvoxl-uvc-server -r 160x120 -f 9.
Next we opened a new terminal and started the voxl-streamer with the default settingsvoxl-streamer -v 0.
In the terminal it shows that it took some default options (w: 1024, h:768, fps: 30), which are not the settings we defined voxl-uvc-server.
As expected, the stream failed to open in VLC.
We now found that when overriding the input pipe specified in the config file to uvc with
voxl-streamer -i uvc -v 0, it takes the correct values as defined in voxl-uvc-server -r 160x120 -f9.
After attempting to open the stream in VLC it crashes after a few frames.
The error messages vary every time we re-start the stream.No. 1:
No. 2:
Do you know how we can solve this issue? -
@wifa799 I think we're going to have to set this up and run some tests to see if we can recreate the issue. Probably won't happen until next week though.
-
Thanks @Eric-Katzfey!
Let me know if I can provide anything from my side. -
@Eric-Katzfey Have you had any chance looking into the issue?
-
@wifa799 Yes, I was finally able to gather some hardware together this morning and it all works as expected, but I have a Pure Thermal 2, not Pure Thermal mini. I know I have a Pure Thermal mini somewhere since that is what I originally tested with so I'll have to try to find it and test that. But it's possible that it isn't the hardware that's the issue. I am using SDK 1.4.0. Which one are you using? What is the output of
voxl-version? I stopped voxl-uvc-server from running automatically withsystemctl disable voxl-uvc-serverand voxl-streamer withsystemctl disable voxl-streamerand then rebooted. Then, from the command line, I started voxl-uvc-server withvoxl-uvc-server -d -r 160x120 -f 9and in another terminal started voxl-streamer withvoxl-streamer -v 0 -i uvc. I then looked at the stream with both QGroundControl 4.3.0 and VLC 3.0.20. What is the output when you usevoxl-uvc-server -d -l? -
-
@wifa799 Are you still having troubles? I got a new PureThermal mini and it also works fine.
-
Hi @Eric-Katzfey, I'm a new student helping out in @wifa799 's project which is why it took a few days for me to respond, sorry for that.
I'll try it out today in the afternoon to be able to also send you the errors I get. So far we haven't gotten it to work. -
Alright I got it running through QGroundControl now. Thank you very much for your help. I still have some open questions though.
In VLC I still cannot connect to it through the rtsp link and also in the VOXL portal I cannot follow the stream (in fact i don't even receive any of the other channels anymore). Do you have any ideas why this might occur and/or how to fix it?Also if I open a third terminal window to check voxl-inspect-services it doesn't show the voxl-uvc-server and the voxl-streamer to be running, even though I can see them working in the other windows and the video also arrives at the QGroundControl. Could this still be an issue?
And lastly I'm curious if it's possible to postprocess the video stream in QGroundControl in real-time for e.g. upscaling or object detection or if you would recommend doing this via gstreamer or something different?
-
@buem799 Glad to hear it is working now. voxl-portal has custom support for various cameras coming through our voxl-camera-server. It does not yet have any support for UVC cameras. Not sure why VLC cannot connect. What version of VLC are you using? voxl-inspect-services only shows the status of services started via systemd. It doesn't show you things that you start on the command line. QGC just shows the video so I don't think you can add those features there. Would you rather do upscaling / object detection directly on VOXL 2 or on a connected computer?
