Posts made by PawelJ

I just want to start by sending a big thanks to the modalai dev team for your great support over the last year. Special thanks to @Alex-Gardner who went above and beyond.

I spent most of the last year putting a system together with your voxl m500 drone to perform autonomous visual inspections, fully onboard with no offboard processes. I just wanted to share the final result with you all. Please let me know if there is a blog or anywhere else where you may like this posted

A few details...the snapdragon is performing the flight control and has the information pipeline (all through the great modal pipe architecture). The snapdragon sends an rtsp stream from the hires camera to a Jetson NX, mounted to the bottom of the drone. The jetson is running inference on the video stream from the snapdragon, and sends detection information back. This is then used to alter the path of the drone and snap a hires photo.

Thanks again to the ModalAI dev team!

https://www.youtube.com/watch?v=nTiyL_NfBCo

Hi all,

I was wondering if there was anything already in place to log the battery information from voxl-inspect-battery to a file. There doesn't seem to be any options to do something like this listed under the -h help flag like I've noticed with a few of your other packages.

Cheers,

Pawel

@m4v3r said in Turn off ESC Beep:

e

Oh that's great news! Thanks a bunch, I'll have to try this out

Thanks for the ideas, I now have options passed the noise cancelling headphones

Hi all,

I was wondering if there was any way of turning off the ESC beeping that turns on after 10 minutes of standby. My current solution is power cycling the voxl through QGroundControl, but this isn't ideal during development.

Cheers,

Pawel

Thanks @Alex-Gardner that did the trick. I can get the cam-server and rtsp server running at the same time now. Is there a way to get voxl-rtsp to run at aspect ratios other than 3:4? When I try to change it I keep getting the following error

/ # voxl-rtsp -s 832x832
voxl-rtsp version 1.0.4
Setting resolution: 832x832
Connected
Started camera 0
Created session
Video track created
in setGainAndExposure: Unable to get camera parameters
rtsp://192.168.86.215:8900/live
Camera Control FIFO starting: /dev/camera0_control
Unable to connect to start session: -19
Unable to delete video track: -19

If I maintain the 3:4 AR it works as expected.

HI @Alex-Kushleyev ,

Thanks for the advice, using the voxl-rtsp server to a specified port and using adb port forwarding worked quite nicely. How would you recommend disabling the hires camera in the voxl-camera-server? I removed it from the config and changed my tflite-server to use the stereo-cameras, but the camera-server throws an error about the J2 port missing.
ERROR: object missing port_J2

Cheers,

Pawel

Is there a way to access the h.264 stream instead of the yuv images from the 4k camera? I am working on sending a video stream from the 4k camera offboard to a jetson NX over the adb port (previous forum post) at a resolution of 832x832. I have a simple message being sent from a server on the voxl to a client on the jetson over the adb port with some adb port forwarding commands. I am working through the process of implementing the h.264 encoding / decoding, however I notice that there is an h264 stream used in voxl-rtsp. Unfortunately I require the voxl-camera-server as I am using vvpx4, and @Alex-Kushleyev mentioned in a prior post of mine that you cannot run the voxl-camera-server and voxl-rtsp at the same time. I am still working my way through the voxl-rtsp code to sort out where the h264 stream is created / accessible from, as I am trying to sort out if there is a way to access the h264 stream without voxl-rtsp, or if that is something created in that service from the raw YUV images.

Thanks, I just wanted to check if I was totally off track. I'll let you know how it goes, the potential of over the air adb updates are very tantalizing

Thanks for the details Eric. Just to make sure I'm following for the adb scenario, would the flow be something like...

-> C application on voxl forwards h264 stream to port x
-> jetson, connected over adb, would run something along the lines of adb forward tcp:x tcp:y
which would forward port x on the voxl to port y on the jetson
-> C or python application on the jetson reads the h264 stream from port y

Cheers

1. Where does the april tag localization change the drone position being sent to the pixhawk to affect the error signal of target-position? I will need to use this position so I can correctly offset my path starting point

The setpoints are are sent to uart_mavlink.c where they are transformed by
geometry_transform_fixed_setpoints_to_local where the april tag transformation matrix is used to convert from fixed to local frame, which is sent off to the pixhawk. If no april tags are used local_frame == fixed_frame

2. Does the px4 expect path in the same reference frame as vio state (ie position in world coordinates, velocities in body frame?)

• px4 expects setpoints in local frame, but offboard figure eight assumes they are set in fixed frame (see above).

Please correct me if I am wrong here.

Cheers

I've gone further down the rabbit hole and see now that the tag detections update the fixed frame, as does the fixed_pose_input sink, which as far as I can tell does not get used and is setup for alternative fixed frame references. I also see now that the setpoints get converted in the uart_mavlink.c file from fixed to local frame, and when no april tags are used this keeps the value the same. This being said, it seems that, when april tags are not used, the setpoints and the vio position are both set in local frame. If I am understanding this correctly, I should be able to set my home position to T_body_wrt_local before sending the home position as a setpoint in offboard mode, but the drone seems to always move to some other location. Can you please clarify what my misunderstaning is in this case, if any?

In vvpx4 I am trying to update the offboard_figure_eight script so that before switching to offboard mode, the home position gets set to the current drone position, and the path gets offset so that it is centered about the current drone state. As a preliminary step I changed the _send_position_in_path function to send the home position instead. I have been using T_body_wrt_local from the vio_manager, after making it externally accessible, but there still seems to be some offset I can't account for that moves the drone along -x and -y from what should be the home position (current state at the time of switch to offboard). Is there a different reference frame I should be using? I noticed that there is a T_body_wrt_fixed that is updated by the apriltag_manager and fixed_pose_input, but this seems to be accounted for in the uart_mavlink_send_fixed_setpoint function that the path waypoints get sent to. Can you please clear up what reference frame would be the correct drone state to use to set my home position to? The current goal is to have the drone hover where it is when it switches to offboard mode, and once this is working, using that same home position to offset the figure8 path, which is centered about the origin by default.

@Alex-Kushleyev thanks for the heads up. You make a good point, I'll switch it over to a bin Thanks!

Hi Eric, thanks for the quick reply. Just a point of clarification, I meant dead thread in regards to the OP not replying after their first post, not in regards to a reply to the question from ModalAI :). What you mentioned about the H264 compression makes sense to save on the bandwith required. The way I see this working would be creating a client on the voxl that subscribes to one of the camera pipes. The client would read in the images and create a buffer where the H264 compression would happen before sending data over usb to the jetson, which will be listening to the stream and doing the decoding to images. I'm not sure about the bandwitch requirements with the compression, but using the adb port would work if it would still allow for the voxl to be connected to a PC for pushing code without much over head (ideally just unplugging the jetson and plugging in to another pc). Do you think this would be the best way to set up the stream, or is there something else you think may work better?

Another question came to mind as I was typing this up....if the jetson is running ubuntu and is connected over the adb port for the video stream, would it also be possible to use the jetson for pushing adb code changes? I guess the question is whether the adb port could be used in two different scenarios, one where the voxl sends the video stream over, and the second where the jetson pushed code over adb to the voxl. This wouldn't be happening at the same time, but more of a question whether anything needs to be changed in how the usb port is used between those two cases on a lower level (drivers etc).

Thank you for all of the ideas. I was able to get the image capture to work by setting my thread to watch for a capture_image flag to start the client, then stop it after the image is saved, instead of threading the callback itself. This let me save an image at a time to text file, then I just scp it off the voxl to parse in python as an RGB image later.

Cheers

Hello,

Can you please clear up how the home position is used in vvpx4? I am looking at the offboard figure eight example and it seems to be sent the same way (same px4 function call uart_mavlink_send_fixed_setpoint, and struct mavlink_set_position_target_local_ned_t) as the offboard path. My confusion is why it needs to be sent if it is just the same message? If any of the three sections in offboard figure eight that send the home position are removed, the drone refuses to switch to offboard mode (before HOME, waiting to switch to offboard mode, and before switching to offboard mode). To clarify my questions...

1. does the px4 differentiate between between the mavlink message containing the home position vs a path setpoint? If not, why does sending the set points of the path in a loop not allow me to switch to offboard mode, like sending the home position in a loop first does?
2. why does the home position need to be sent, specifically in the three locations (listed above) to be able to switch to offboard mode?
3. Is there a limit to how many home position messages need to be sent to switch to offboard mode? In the example it appears to be set for 2 seconds, however I believe that is to give the drone enough time to return to the hardcoded path start (0, 0, FLIGHT_ALTITUDE).

I am working on updating the positions of a custom path in vvpx4 so that it is updated to be offset by the current position prior to switching to offboard mode. At first I have been using T_body_wrt_local from the vio_manager.c, but I notice that this value does not get changed with april tag detections. Following the chain, I see...
--> the tag_manager calls geometry when we detect new tags and have the correct config values set here
--> we get our updated transform from the pose_filter, that updates the pointer to filter_local_to_fixed with what we calculted as new_T_local_wrt_fixed, which I believe is the new drone state I want to use, set here
--> global variables get updated with rc_pose_filter_fetch where the pointer to T_local_wrt_fixed gets overwritten with the value we just pushed to the filter_local_to_fixed, here

Now when we get the drone state from the vio_manager it only calls geometry_get_T_body_wrt_local, and I do not see where the drones state is updated with the new april tag offset. This could just be a misunderstanding on my part in how the system should work. The way I expect it to work is that the drone initializes it's state to be relative to starting position as the origin. This sets the global frame and is the reference frame the pixhawk expects drone state and targets to be set in. If this were the case, I would expect this to shift the state of the drone when a tag is detected (probably with some filtering on the px4 side that I'm also missing, to avoid the dicsontinuous jump in position). However, it appears that the drone state is set with geometry_get_T_body_wrt_local, and not the new fixed frame set by the april tag.

Can you please clarify:

1. Where does the april tag localization change the drone position being sent to the pixhawk to affect the error signal of target-position? I will need to use this position so I can correctly offset my path starting point
2. Does the px4 expect path in the same reference frame as vio state (ie position in world coordinates, velocities in body frame?)

Hello,

I am looking at connecting the voxl m500 reference drone to feed 832x832 RGB images at 30fps to the jetson NX that will be using the feed to run a localization network. The Jetson would only occasionally send small packets back to the voxl, possibly even a T/F, depending on the detection/localization. I noticed a similar post here that seems to be dead that I didn't want to hijack, but I'll link in case their is activity later on. I'm looking for a little bit of guidance to make some intelligent choices in regards to the hw and sw setup. To answer some of the questions asked from modalai there, I believe that I will need to use UART or SPI to get the bandwidth I need. The jetson is essentially running a full version of ubuntu with it's own wifi card, so I do not think it really matters which is the client / host, but I am open to your suggestion as to what may be a better implementation.

I've been working on adding some vio logging to the vvpx4 repo. I'm branched off the most recent master version as of July 30 2021. The idea is to be able to track the drones location, specifically when running offboard mode (figure8 in this case). I'm using the system date-time stamp to name the log files created, and the datetime returned changes depending how vvpx4 is run. If I let it run in the background with systemctl restart voxl-vision-px4 It keeps getting 1970-01-01_-4-00-08 or 1970-01-01_-4-00-09 as the date-time stamp, but if I run the service manually with voxl-vision-px4 to see printouts (note not running with the debug flag) I get the datetime I expect. The issue here is that my logs keep getting overwritten unless I am running the vvpx4 package manually. Just a quick note that I am subtracting 4 hours to correct for my timezone (can be seen in screenshots below).

The changes are fairly minor and only extend to the vio_manager and offboard_figure8 scripts. I can email the scripts if it will help. For now I'll add some screenshots of the diff from master.

vio_manager.c
I made T_body_wrt_local and usec externally accessible from the vio_manager.c.

offboard_figure_eight.h
I added some variables to the figure8 header for the file handles

offboard_figure_eight.c
I instantiate my date-timestamp structs and the now externally facing T_body_wrt_local and usec variables from the vio_manager. Then I use the datetime to create the filename.

I have the file open outside the main HOME loop of offboard_figure_eight, and I write to file during each loop following (not armed and in offboard mode, sending home position before offboard mode, and in offboard mode).

tldr; why is the system date-time returned different when vvpx4 is running as voxl-vision-px4 vs systemctl restart voxl-vision-px4?