Connecting VOXL to Jetson NX
-
Hello,
I am looking at connecting the voxl m500 reference drone to feed 832x832 RGB images at 30fps to the jetson NX that will be using the feed to run a localization network. The Jetson would only occasionally send small packets back to the voxl, possibly even a T/F, depending on the detection/localization. I noticed a similar post here that seems to be dead that I didn't want to hijack, but I'll link in case their is activity later on. I'm looking for a little bit of guidance to make some intelligent choices in regards to the hw and sw setup. To answer some of the questions asked from modalai there, I believe that I will need to use UART or SPI to get the bandwidth I need. The jetson is essentially running a full version of ubuntu with it's own wifi card, so I do not think it really matters which is the client / host, but I am open to your suggestion as to what may be a better implementation.
-
I don't think the other thread is dead as it offered some reasonable choices. You have those same options available but need to consider your bandwidth requirements. An 832x832 image is 692,224 pixels. If each RGB pixel is 24 bits then you have a total of 16,613,376 bits per image. At 30 fps that's 498,401,280 bps or roughly 500Mbps. A UART or SPI bus isn't going to be able to deal with that. Can you compress those images into JPEG, or a h.264 stream? If not then USB 3.0 is really your only route. The Voxl has a USB 3.0 OTG port available but that is also the adb port so you have to decide if that works for you. Most likely you would need to figure out how to make the Jetson NX a USB slave that enumerates as a network interface with the Voxl as the USB master. For example: https://en.wikipedia.org/wiki/Ethernet_over_USB
-
Hi Eric, thanks for the quick reply. Just a point of clarification, I meant dead thread in regards to the OP not replying after their first post, not in regards to a reply to the question from ModalAI :). What you mentioned about the H264 compression makes sense to save on the bandwith required. The way I see this working would be creating a client on the voxl that subscribes to one of the camera pipes. The client would read in the images and create a buffer where the H264 compression would happen before sending data over usb to the jetson, which will be listening to the stream and doing the decoding to images. I'm not sure about the bandwitch requirements with the compression, but using the adb port would work if it would still allow for the voxl to be connected to a PC for pushing code without much over head (ideally just unplugging the jetson and plugging in to another pc). Do you think this would be the best way to set up the stream, or is there something else you think may work better?
Another question came to mind as I was typing this up....if the jetson is running ubuntu and is connected over the adb port for the video stream, would it also be possible to use the jetson for pushing adb code changes? I guess the question is whether the adb port could be used in two different scenarios, one where the voxl sends the video stream over, and the second where the jetson pushed code over adb to the voxl. This wouldn't be happening at the same time, but more of a question whether anything needs to be changed in how the usb port is used between those two cases on a lower level (drivers etc).
-
@PawelJ If you can compress the video stream then that opens up more possibilities. You can specify a bandwidth target with h264. This would allow you to consider the USB 2.0 port (Although that is host mode only (i.e. master)). You could also consider UART if you bring the bandwidth down to 1 or 2Mbps. As far as running adb on Jetson I think that is possible. Then you could use adb with Voxl from the Jetson. But I'm not sure how you would stream the video if adb was connected. You would have to investigate the options with adb but there may be a network proxy option that you could use for this.
-
Thanks for the details Eric. Just to make sure I'm following for the adb scenario, would the flow be something like...
-> C application on voxl forwards h264 stream to port x
-> jetson, connected over adb, would run something along the lines ofadb forward tcp:x tcp:y
which would forwardport xon the voxl toport yon the jetson
-> C or python application on the jetson reads the h264 stream fromport yCheers
-
@PawelJ That seems like a reasonable thing to try, but we haven't tried this before.
-
Thanks, I just wanted to check if I was totally off track. I'll let you know how it goes, the potential of over the air adb updates are very tantalizing
