Python, OpenCV, MAVLink, Docker, and cameras how to use all?
-
Hello! So I work on a ModalAI m500 developement drone. I've used Docker to run python programs along with MAVSDK. My next goal is to create a program that detects faces or AprilTags or any specified object and make the drone follow it while the object/person moves, however I don't know how to stream what the camera sees into my Pyhton program using OpenCV. Considering my program is running inside a Docker, how can I get real time images from any of the cameras on the drone, make the stream get into the Docker, make the Python program get that stream, process it and make the drone react to the processing output?
Thank you!
-
Not Modalai, but this is what I used. When running the docker container, make sure you run it with the --device tag. This will let you add devices such as cameras you can then access with opencv. For instance, if I use the gimbal camera, I would add to the "docker run" command the "--device=/dev/video2." I would then use opencv like "cap = cv2.VideoCapture(2)" which will try to access the camera at /dev/video2. I'm uncertain whether this process will work for the other cameras, but theoretically as long as you find the correct folders where the camera devices are stored, you should be able to connect to them
-
Hello! So I work on a ModalAI m500 developement drone. I've used Docker to run python programs along with MAVSDK. My next goal is to create a program that detects faces or AprilTags or any specified object and make the drone follow it while the object/person moves, however I don't know how to stream what the camera sees into my Pyhton program using OpenCV. Considering my program is running inside a Docker, how can I get real time images from any of the cameras on the drone, make the stream get into the Docker, make the Python program get that stream, process it and make the drone react to the processing output?
Thank you!
@Alfonso-Ponce If you want to use any of the standard cameras then you need to get the frames from voxl-camera-server. It is based on our "ModalAI Pipe Architecture" (MPA) that uses standard Linux pipes to move data between processes. You can make an MPA client application in your Docker and receive the frames there. You will likely need to use the "-v" flag when launching your container to remap the pipe names outside of the Docker to new names inside of the Docker (e.g. -v <pipe name>:<container pipe name>)
-
@Alfonso-Ponce If you want to use any of the standard cameras then you need to get the frames from voxl-camera-server. It is based on our "ModalAI Pipe Architecture" (MPA) that uses standard Linux pipes to move data between processes. You can make an MPA client application in your Docker and receive the frames there. You will likely need to use the "-v" flag when launching your container to remap the pipe names outside of the Docker to new names inside of the Docker (e.g. -v <pipe name>:<container pipe name>)
@Eric-Katzfey , would you be able to share a minimal code example (preferably in python) that retrieves a single frame from the high-res camera? The minimal code would be running in a Docker container running on the VOXL.
-
I'm going to assume you know about the Docker examples at:
https://gitlab.com/voxl-public/voxl-docker-imagesWill any of those help?
-Todd
Reading along? Create a free ModalAI Forum account to join in.
With an account you can reply, ask your own question, get an email when a ModalAI engineer answers, and mark the reply that solved it. Your place in each thread is saved between visits.
Questions about VOXL, Flight Core, ESCs and ModalAI drones are answered here by the engineers who build them.
Register Login