I/O Python Interfacing

Chad Sweet

Can you send the screenshot from the flashing process? Sometimes you need to run with 'sudo' or manually install via 'sudo fastboot'

if you type 'sudo fastboot devices' what do you see?

PawelJ

Hi @Chad-Sweet ,

Thanks for replying so quickly, I really appreciate it as this has been blocking for my work.

When I check for fastboot devices I see

(test) pjaworsk@ABR04:~/src/voxl/voxl-vision-px4$ sudo fastboot devices
[sudo] password for pjaworsk: 
534625ca	fastboot

modaltb

In the installer folder that gets unzipped, there's a system-image directory, which has a flash_build_apps.shscript.

This is getting stuck on:

while [ "$(fastboot devices)" == "" ]
	do
		echo "[INFO] Waiting for fastboot..."
		sleep 2
	done

Can you please try to run the installer with sudo? e.g. sudo ./install.sh

Something with permissions can get funky with fastboot and running with sudo seems to get through the issue normally.

PawelJ

Thanks, that got the install through. You can ignore the panicked email I sent out earlier I was worried I had bricked the device and would have to wait for a usb expansion board to ship.

That fixed the issue with VIO local position ned and it is now visible in QGC again. I still seem to have the issue with my radio transmitter. QGC vocalizes the commands as usual, but I can't seem to arm the motors. When I go to the radio page of QGC it gives me this error Screenshot from 2021-05-05 12-59-57.png
It looks like this could have to do with the calibration going off the docs, but if I try to recalibrate the same error comes up.

PawelJ

It looks like I spoke too soon. The VIO data was sent after the initial reinstall. Once I tried compiling and building the dev branch of voxl-vision-px4 (to use the polynomial path planning) it stopped sending to QGC again. I tried installing the master branch to fix it, but it still isn't sending. Following the same steps I listed above that I followed from this forum post, The voxl-test-vision-lib script functions as expected and voxl-vision-px4 voxl-camera-server and voxl-qvio-server are running.

~ # systemctl status voxl-vision-px4
● voxl-vision-px4.service - voxl-vision-px4
   Loaded: loaded (/usr/bin/voxl-vision-px4; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-05 19:45:52 UTC; 14s ago
  Process: 11248 ExecStartPre=/bin/sleep 2 (code=exited, status=0/SUCCESS)
 Main PID: 11259 (voxl-vision-px4)
   CGroup: /system.slice/voxl-vision-px4.service
           └─11259 /usr/bin/voxl-vision-px4

May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting fixed pose input
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting vio manager
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting apriltag manager
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting voa manager
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: starting offboard figure eight
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Init complete, entering main loop
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Connected to voxl-qvio-server
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: done updating transforms to use imu: imu1
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: Added new UDP connection to 192.168.86.176
May 05 19:45:55 apq8096 voxl-vision-px4[11259]: PX4 Connected over UART with sysid 1

~ # systemctl status voxl-qvio-server
● voxl-qvio-server.service - voxl-qvio-server
   Loaded: loaded (/usr/bin/voxl-qvio-server; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-05 19:23:52 UTC; 22min ago
 Main PID: 6081 (voxl-qvio-serve)
   CGroup: /system.slice/voxl-qvio-server.service
           └─6081 /usr/bin/voxl-qvio-server

May 05 19:23:52 apq8096 voxl-qvio-server[6081]: LNX_IA64 supported? 1
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: WINDOWS supported? 0
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: AR ERROR: arFileOpen(): Failed to open file: //vislam/Configu....xml
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: FASTCV: fcvAvailableHardware Linux
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: mempool cur block size 307200, new block size 307200
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: Please ignore the error about Configuration.SF.xml above. ^^^
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: It's an optional file, and should be a warning not an error
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: waiting for imu
May 05 19:23:52 apq8096 voxl-qvio-server[6081]: waiting for cam
May 05 19:44:32 apq8096 voxl-qvio-server[6081]: WARNING: output data thread fell behind
Hint: Some lines were ellipsized, use -l to show in full.

~ # systemctl status voxl-camera-server
● voxl-camera-server.service - voxl-camera-server
   Loaded: loaded (/usr/bin/voxl-camera-server; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-05-05 19:45:33 UTC; 56s ago
 Main PID: 10989 (voxl-camera-ser)
   CGroup: /system.slice/voxl-camera-server.service
           └─10989 /usr/bin/voxl-camera-server -c /etc/modalai/voxl-camera-server.conf

May 05 19:45:34 apq8096 bash[10989]: AEAlgo     : mvcpa
May 05 19:45:34 apq8096 bash[10989]: mvcpa_filter_size    : 2
May 05 19:45:34 apq8096 bash[10989]: mvcpa_exp_cost       : 0.7500
May 05 19:45:34 apq8096 bash[10989]: mvcpa_gain_cost      : 0.2500
May 05 19:45:34 apq8096 bash[10989]: mvcpa_histogram      : false
May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/hires_preview/ channel: 0
May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/stereo/ channel: 1
May 05 19:45:34 apq8096 bash[10989]: Creating pipe: /run/mpa/tracking/ channel: 2
May 05 19:45:34 apq8096 bash[10989]: tracking
May 05 19:45:34 apq8096 bash[10989]: Camera Width: 640, Height: 480, Format: 291 not supported!

However, my voxl-imu-server status is different, and I'm not sure if this is as it's expected to be. It also remains like this after I restart the service

~ # systemctl status voxl-imu-server
● voxl-imu-server.service - voxl-imu-server
   Loaded: loaded (/usr/bin/voxl-imu-server; enabled; vendor preset: enabled)
   Active: activating (start-pre) since Wed 2021-05-05 19:49:03 UTC; 1s ago
  Process: 11969 ExecStart=/usr/bin/voxl-imu-server (code=exited, status=255)
 Main PID: 11969 (code=exited, status=255);         : 11975 (sleep)
   CGroup: /system.slice/voxl-imu-server.service
           └─control
             └─11975 /bin/sleep 2

I've included a list of my voxl and modal packages since the update

/ # opkg list-installed | grep "voxl"
libvoxl_io - 0.5.4
voxl-camera-server - 0.5.7
voxl-dfs-server - 0.0.7
voxl-docker-support - 1.1.1
voxl-gphoto2 - 0.0.5
voxl-hal3-tof-cam-ros - 0.0.5
voxl-imu-server - 0.7.8
voxl-modem - 0.11.0
voxl-mpa-tflite-server - 0.0.2
voxl-mpa-tools - 0.1.6
voxl-nodes - 0.1.3
voxl-qvio-server - 0.2.1
voxl-rtsp - 1.0.3
voxl-streamer - 0.2.1
voxl-suite - 0.3.4
voxl-utils - 0.6.0
voxl-vision-px4 - 0.8.1
voxl-vpn - 0.0.3
/ # opkg list-installed | grep "modal"
libmodal_json - 0.3.4
libmodal_pipe - 1.7.9
modalai-vl - 0.1.3

Just for completeness, I've also included my voxl-vision-px4.conf

/**
 * VOXL Vision PX4 Configuration File
 *
 */
{
        "qgc_ip":       "192.168.86.208",
        "en_localhost_mavlink_udp":     true,
        "en_secondary_qgc":     false,
        "secondary_qgc_ip":     "192.168.1.214",
        "qgc_udp_port_number":  14550,
        "localhost_udp_port_number":    14551,
        "udp_mtu":      512,
        "en_vio":       true,
        "en_voa":       true,
        "en_send_vio_to_qgc":   true,
        "en_send_voa_to_qgc":   false,
        "en_set_clock_from_gps":        true,
        "en_force_onboard_mav1_mode":   true,
        "en_reset_px4_on_error":        true,
        "qvio_auto_reset_quality":      0.00050000002374872565,
        "en_adsb":      false,
        "adsb_uart_bus":        7,
        "adsb_uart_baudrate":   57600,
        "px4_uart_bus": 5,
        "px4_uart_baudrate":    921600,
        "offboard_mode":        "figure_eight",
        "follow_tag_id":        0,
        "en_apriltag_fixed_frame":      false,
        "fixed_frame_filter_len":       5,
        "en_transform_mavlink_pos_setpoints_from_fixed_frame":  false
}

PawelJ

I've sent a few messages and listed a few problems above, so I just want to reiterate the current problem that's lead me here to stay on track.

I'm trying to set a custom path and record the VIO position output (local_position_ned) from the named pipe while the path is being followed. My plan is to use the dev branch of voxl-vision-px4 and incorporate something like this example from libmodal-pipe to do the reading. This would be incorporated into the path script so it gets run whenever the offboard mode is following the path. I still have to look into the polynomial path planning code so can't speak on that yet.

For the use case I imagine I would still have manual control from the transmitter so I can manual set the kill switch and take over control if anything happens. I also use this to enable offboard mode. Given the problems above, I can't use the transmitter to set the mode on the drone, so I can't test out any offboard flight. Since I'm also embedding the vio feedback recording in that path, I can't run that without switching to offboard mode.

I imagine the issue of VIO not being sent to QGC is related to the transmitter communicating with QGC, but not with the drone (no state change or motor arming).

Hi Pawel,
It's hard to tell exactly what's going on from these cutouts, would you be available for a short 30ish minute call where you can explain exactly what the issue is and I can help real time to figure out what's wrong, I think it'll be much quicker than going back and forth here.

PawelJ

Hi @Alex-Gardner ,

That would be great! We can do it over a video chat if that works. Feel free to send me a link at my email pawel.jaworski@appliedbrainresearch.com.

Thank you

PawelJ

Just posting an update after some discuission with @Alex-Gardner. Thank you again, you were incredibly helpful in understanding the overall voxl comm pipeline.

Solution for Python I/O
The easiest method to do this was to run a modified vvpx4 that would record vio position based on a flag. Using the dev branch, I modified process_new_vio_data() in vio_manager found here. I just added a snippet that would write T_body_wrt_local.d to a file in /data/my_folder. I have the flag set to start and stop logging when an offboard flight mode is started and stopped, by setting a flag in the HOME block of the path.

Solution for QGroundControl and Transmitter Connection Problems
I had to reset the px4 parameters and recalibrate. This fixed my connectivity issues and I can now change modes and arm the motors through my transmitter. I can also see VIO data in QGC as well. The process I followed for this portion was based off the solution to this forum post and the steps in the docs here:

reset px4 parameters to firmare's default
load the m500_4s_config.params - the instructions say to do this twice, however I think this is outdated as it didn't seem necessary for me, as the second time I went to upload the params I got a "no changes made" type notification
load the mavlink_serial_ports.params helper config
run through the sensor calibration steps on QGC, I used a flat sheet of mdf on a large turntable to keep rotate the drone. Standing on its 4 legs and upside-down was fine, but for the other 4 orientations I would balance it on two legs. It would stay stable, but would be slightly off from a perfect right angle. From the PX4 docs it looks like this is fine for accelerometer calibration since it uses a least squares fit.

Unresolved Problems
There is still something wrong here based on how the drone is flying, and I cannot tell whether it's related to the calibration or some missing parameters. It mostly flies fine in manual mode, maybe a little bit of drift, but it seems about the same as it was when I received the drone calibrated. In position control it starts to drift when in flight, and when stationary the VIO output drifts as well. Here is a video I took of it "in the field." Appologies for the cellphone video, I had no way of screencapturing at the time. You can see the drone is stationary and it's still drifting. I don't think it has to do with trackable features, as this setup worked with when I first got the drone. I also took a video of the flight, going from manual, to position, to offboard (I say in the video when I change modes). You can see the drift here as I try to keep the drone centered in the net. The vio drift video with the drone stationary is with my modified vvpx4 where the vio position is saved when in offboard mode, so this should make no changes. However, I also tested with the master branch of vvpx4, which is the second video with the drone flying. Since it happens on the master branch, I think the problem is related to the QGC/PX4 setup and calibration.

Q1 One possible problem is that the compass calibration says to stay away from metal, so possibly the 12" turntable was causing some interference, but I don't think it is likely because of the thick mdf on top of it. One point I am unsure of is whether the gps is required for this, as where I calibrate has no gps signal. Can you confirm whether gps is required for this step (I imagine not since you offer a gps free version)?

Q2 One other possible problem is that when I look at my px4 parameters, I do not see the PWM_MIN parameter listed in the docs when testing if the parameters were uploaded, which could be a sign of some missing px4 configs. From a fresh QGC/PX4 setup, is there something other than the mavlink_serial_ports and m500_4s_config that is required?

PawelJ

I reran the calibration outdoors with gps signal and far from any metal. I also raised the drone to be ~3-4' from the metal turntable below it. This improved the calibration quite a bit and the drone is not drifting as much as before. However, there still seems to be more drift in the VIO than there was when I first received the drone. I still don't have the PWM_MIN parameter that the docs say I should check for the assure the PX4 parameters loaded properly. I see that this parameter is in the custom2.params file, but the note in the document says it turns off the gps.

Can you please explain the steps you go through during your calibration process? Should I also be loading this custom2.params file? If so, does that mean that gps is not used when in VIO (no sensor fusion etc)?