Hi @K-Stute, sorry for the late reply .
Anyway, I connected the camera over the usb connection on the 5G modem, you need to be sure to have the proper cable (the standard one of the realsense is fine) and that the connection is over a usb 3.0 (only one of the two port of the modem works), otherwise everything works quite out of the box if you run everything on docker container (haven't tried without a container).
Hi @Moderator and @Kashish-Garg, we have noticed a similar behavior involving a discrepancy between the qvio server and /mavros/local_position/pose on the z-axis. We have attempted to resolve this by disabling and re-enabling the barometer sensor, but the issue persists. The inconsistency does not always occur and does not appear to be related to aggressive flight behaviors.
Hi @Alex-Kushleyev @thomas sorry for the late reply, but the problem seems to be solved when we changed the power module which probably was causing the failure.
Hi @thomas,
I createad a custom ai_segmentation_t struct as
#define MASK_HEIGHT 480
#define MASK_WIDTH 640
typedef struct ai_segmentation_t
{
uint32_t magic_number;
int64_t timestamp_ns;
uint32_t class_id;
int32_t frame_id;
char class_name[BUF_LEN];
char cam[BUF_LEN];
bool first_segmentation;
bool last_segmentation;
float class_confidence;
float detection_confidence;
float x_min;
float y_min;
float x_max;
float y_max;
int8_t mask[MASK_HEIGHT * MASK_WIDTH]; // 2D array for segmentation mask
} __attribute__((packed)) ai_segmentation_t;
which are then written down to the pipe as (main.cpp file of the voxl-tflite-server)
std::vector<ai_segmentation_t> detections;
if (!inf_helper->postprocess_seg(output_image, detections, last_inference_time))
continue;
if (!detections.empty())
{
for (unsigned int i = 0; i < detections.size(); i++)
{
pipe_server_write(DETECTION_CH, (char *)&detections[i], sizeof(ai_segmentation_t));
}
}
new_frame->metadata.timestamp_ns = rc_nanos_monotonic_time();
pipe_server_write_camera_frame(IMAGE_CH, new_frame->metadata, (char *)output_image.data);
}
And retrieved by the ai_detection_interface.h in the callback function of voxl-mpa-to-ros
static void _helper_cb(__attribute__((unused)) int ch, char *data, int bytes, void *context)
{
AiDetectionInterface *interface = (AiDetectionInterface *)context;
if (interface->GetState() != ST_RUNNING)
return;
ros::Publisher &publisher = interface->GetPublisher();
voxl_mpa_to_ros::AiSegmentation &obj = interface->GetSegMsg();
std::vector<voxl_mpa_to_ros::AiSegmentation> obj_array;
voxl_mpa_to_ros::AiSegmentationArray obj_arrayMsg;
int n_packets = bytes / sizeof(ai_segmentation_t);
ai_segmentation_t *detections = (ai_segmentation_t *)data;
bool first_segmentation = false;
bool last_segmentation = false;
for (int i = 0; i < n_packets; i++)
{
if (detections[i].magic_number != AI_DETECTION_MAGIC_NUMBER)
return;
// publish the sample
obj.timestamp_ns = detections[i].timestamp_ns;
obj.class_id = detections[i].class_id;
obj.frame_id = detections[i].frame_id;
obj.class_name = detections[i].class_name;
obj.cam = detections[i].cam;
obj.class_confidence = detections[i].class_confidence;
obj.detection_confidence = detections[i].detection_confidence;
obj.x_min = detections[i].x_min;
obj.y_min = detections[i].y_min;
obj.x_max = detections[i].x_max;
obj.y_max = detections[i].y_max;
obj.first_segmentation = detections[i].first_segmentation;
obj.last_segmentation = detections[i].last_segmentation;
if (detections[i].first_segmentation)
{
first_segmentation = true;
}
if (detections[i].last_segmentation)
{
last_segmentation = true;
}
last_segmentation = true;
first_segmentation = true;
obj.mask = std::vector<int8_t>(detections[i].mask, detections[i].mask + sizeof(detections[i].mask) / sizeof(detections[i].mask[0]));
// add the
obj_array.push_back(obj);
}
obj_arrayMsg.segmentation_array = obj_array;
if (n_packets > 0 && publisher.getNumSubscribers() > 0 && first_segmentation && last_segmentation)
{
publisher.publish(obj_arrayMsg);
}
return;
}
and inside the ai_detection_interface has the ai_segmentation and ai_segmentation_array defined as (same as the tflite-server):
typedef struct ai_segmentation_t
{
uint32_t magic_number;
int64_t timestamp_ns;
uint32_t class_id;
int32_t frame_id;
char class_name[BUF_LEN];
char cam[BUF_LEN];
bool first_segmentation;
bool last_segmentation;
float class_confidence;
float detection_confidence;
float x_min;
float y_min;
float x_max;
float y_max;
int8_t mask[MASK_HEIGHT * MASK_WIDTH]; // 2D array for segmentation mask
} __attribute__((packed)) ai_segmentation_t;
typedef struct ai_array_segmentation_t {
std::vector<ai_segmentation_t> segmentations;
} __attribute__((packed)) ai_array_segmentation_t;
The problem is that the model runs and the output of the detection makes sense :
The usage of the GPU is:
[Name Freq (MHz) Temp (C) Util (%)
-----------------------------------
cpu0 1075.2 67.1 33.30
cpu1 1075.2 65.5 26.08
cpu2 1075.2 65.9 26.77
cpu3 1075.2 65.9 26.34
cpu4 710.4 67.1 16.08
cpu5 710.4 65.5 25.00
cpu6 710.4 65.5 23.67
cpu7 844.8 67.1 6.61
Total 67.1 22.98
10s avg 22.78
-----------------------------------
GPU 587.0 67.1 97.92
GPU 10s avg 49.08
-----------------------------------
memory temp: 65.0 C
memory used: 1125/7671 MB
-----------------------------------
Flags
CPU freq scaling mode: auto
Standby Not Active
-----------------------------------]
However, when I attempt to retrieve the segmentation from the voxl-mpa-to-ros the voxl2 reboot, even though some messages are received. While I might expect a crash at the code level or a segmentation fault, a total reboot occurs, which can create a few problems, especially when flying.
Also even tought it occurs more sporadically, the voxl2 also reboot by simply running the model and displaying on the portal the segmentations. However the voxl-inspect-cpu does not complain about overheating neither I notice any extra memory usage.
Any help is welcome! 
Hello everyone, I'm currently dealing with an issue while running a custom segmentation model on the tflite server. The board is maintaining an approximate inference time of 5Hz, which is good. However, I'm facing problems when trying to pass segmentation masks over pipes to be retrieved by the voxl-mpa-to-ros pipeline.
I've implemented a solution using a new data structure for the masks, which are larger than normal bounding boxes. I publish these masks to the pipe, and on the receiving end, the mpa_to_ros node reads and forwards the mask as a custom message. This process worked well with SDK 0.9.5, but after upgrading to SDK 1.1.2, the board crashes and reboots when attempting to subscribe to the tflite data (I can get few samples but then it crash). The code remains the same, and I've also tested changing the buffer, but the issue persists.
Interestingly, subscribing to the camera portal works fine, but subscribing to the pipe causes the crash. I'm wondering if it could be a problem with the SDK, perhaps an issue with memory management causing the board to crash and reboot. Or could it be a power supply problem?
Has anyone else experienced a similar issue after upgrading to SDK 1.1.2, or does anyone have insights into what might be causing the problem? Any suggestions for troubleshooting or potential solutions would be greatly appreciated. Thank you!
Hi,
after experimenting, we found that the problem was a RealSense external camera that we plugged in.
It was connected through USB via the 5G module. Probably, this was causing a short circuit or overload of the board. Anyway, modifying the connection cable so that only the data goes through the board while it is powered separately seems to solve the problem.
I'm currently facing a challenge with my VoXL2 Flight Deck (with power module v3), where it seems to reboot due to a power limitation issue. I'm running SDK version 1.1.2 and trying to execute a segmentation model converted to TFLite. The model runs successfully at 5Hz, but the problem arises when I observe a significant increase in power consumption, particularly the current, when I try to run anything else.
Here are the power metrics before and after starting the model:
Before Starting Model:
Voltage | Charge | Current
14.70V | 100% | 0.63A
After Starting Model:
Voltage | Charge | Current
14.70V | 100% | 0.91A
The current spikes to 0.8A, with peaks of 0.9A. When I also run the voxl-mpa-to-ros node and another custom node, the power change is significant enough to trigger a reboot of the drone. I've experimented with the skip_n_frames parameter to achieve a more stable result, but the board seems to be operating at its limit. Any attempt to surpass 1.0A, easily triggered by subscribing to ROS topics or running basic logging scripts, results in a reboot, rendering it unsuitable for flying.
Key observations:
GPU usage is at its limit.
CPU and temperature are within acceptable ranges.
I'm seeking guidance on how to address this power limitation. Are there specific parameters that can be configured or ways to modify the power limit of the board to handle the increased load without triggering reboots?
Any insights or suggestions would be greatly appreciated. Thank you in advance!
Hi @Eric-Katzfey eventually we managed to solve the problem by resetting to default the parameter of px4 and performing the calibration from qgc
@Eric-Katzfey
Hi Eric, I tried using QGroundControl. The accelerometer and gyro have been successfully calibrated, but I encountered an error stating "no mags" when attempting to calibrate the compass and then nothing happen. Do you have any ideas about what could be causing this issue?
@Alex-Kushleyev sure are those :
Calibration File Status:
Missing /data/modalai/voxl-imu-server.cal
Missing /data/modalai/opencv_tracking_intrinsics.yml
Missing /data/modalai/opencv_stereo_front_intrinsics.yml
Missing /data/modalai/opencv_stereo_front_extrinsics.yml
Missing /data/modalai/opencv_stereo_rear_intrinsics.yml
Missing /data/modalai/opencv_stereo_rear_extrinsics.yml
Missing /data/px4/param/parameters_gyro.cal
Missing /data/px4/param/parameters_acc.cal
Missing /data/px4/param/parameters_level.cal
Missing /data/px4/param/parameters_mag.ca
The camera ones and the IMU I can recover them by using the calibration tools but I haven't found a solution for the px4 ones