@Alex-Kushleyev I've got this working using a custom ModalAI pipe sending an RGB image (I tried YUV because I thought that was the expected input but looking through the source code I saw the vixl-streamer could accept RGB as well).
I am unfortunately only successful with a 480p image and nothing at a higher resolution. I do not get any error messages writing to the pipe with a larger image size and I don't see the voxl-streamer receive an initial frame. Do you have thoughts as to why I'm not seeing any output from voxl-streamer and not getting an error when writing to the pipe?
I've attached the logs from my application and from running /usr/bin/voxl-streamer -v 0.
My configuration file is
{
"input-pipe": "ros_to_voxl_streamer",
"bitrate": 10000000,
"decimator": 1,
"port": 8900,
"rotation": 0
}
The logs from the voxl-streamer can be found here for 480p images and here for 720p images. The logs from my application can be found here for the 480p image output and here for the 720p image output.
When I write to the pipe my function is
bool image_pipe::write_frame(cv::Mat frame_bgr)
{
namespace time = std::chrono;
log_verbose(fmt::format("Input frame has size [width={}, height={}]",
frame_bgr.size().width,
frame_bgr.size().height));
static auto first_frame = true;
static std::size_t num_frame_write_errors = 0;
static constexpr std::size_t num_acceptable_frame_write_errors = 10;
// Resize the image to the output size specified
cv::resize(frame_bgr, frame_bgr, img_size);
log_verbose(fmt::format("Resized input frame to size [width={}, height={}]",
frame_bgr.size().width,
frame_bgr.size().height));
cv::Mat frame_rgb;
cv::cvtColor(frame_bgr, frame_rgb, cv::COLOR_BGR2RGB);
// If this is the first frame we've sent then configure the
// camera metadata
if (first_frame)
{
camera_mdata.height = static_cast<std::int16_t>(img_size.height);
camera_mdata.width = static_cast<std::int16_t>(img_size.width);
camera_mdata.size_bytes = static_cast<std::int32_t>(frame_rgb.total()) *
static_cast<std::int32_t>(frame_rgb.elemSize());
camera_mdata.stride = static_cast<std::int32_t>(frame_rgb.step[0]);
log_info(fmt::format("First frame written to image_pipe object. Metadata:\n{}",
describe_camera_pipe_metadata()));
first_frame = false;
}
camera_mdata.timestamp_ns = time::duration_cast<time::nanoseconds>(
time::steady_clock::now().time_since_epoch()).count();
log_verbose(fmt::format("Sending image through pipe with metadata\n{}",
describe_camera_pipe_metadata()));
const auto* frame_data = reinterpret_cast<const void*>(frame_rgb.data);
const auto ret = pipe_server_write_camera_frame(0, camera_mdata, frame_data);
if (ret != 0)
{
if (num_frame_write_errors++ == num_acceptable_frame_write_errors)
{
throw std::runtime_error("Maximum number of consequtive frame write errors reached");
}
log_warning(fmt::format("Was not able to write a frame to the pipe. There are {} more error(s) left before failure",
num_acceptable_frame_write_errors - num_frame_write_errors));
return false;
}
num_frame_write_errors = 0;
if (++camera_mdata.frame_id % 100 == 0)
{
log_verbose(fmt::format("Camera metadata:\n{}",
describe_camera_pipe_metadata()));
}
return true;
}
Note that the calls to log_ functions are what generates the above-linked log files.
and the construction of the pipe occurs here:
static inline pipe_info_t make_pipe_info(const std::string_view pipe_name)
{
pipe_info_t info; // = PIPE_INFO_INITIALIZER;
std::strcpy(info.name, pipe_name.data());
std::strcpy(info.type, "camera_image_metadata_t");
std::strcpy(info.server_name, "ros-voxl-streamer");
info.size_bytes = 2 * MODAL_PIPE_DEFAULT_PIPE_SIZE;
return info;
}
image_pipe::image_pipe(std::string pipe_name, cv::Size _img_size)
: voxl_pipe_info(make_pipe_info(pipe_name)),
img_size(_img_size)
{
// Configure the camera_mdata member
camera_mdata.magic_number = CAMERA_MAGIC_NUMBER;
camera_mdata.exposure_ns = -1;
camera_mdata.gain = -1;
camera_mdata.format = IMAGE_FORMAT_RGB; // IMAGE_FORMAT_YUV420;
camera_mdata.frame_id = 0;
camera_mdata.framerate = 5;
const auto ret = pipe_server_create(0, voxl_pipe_info, 0);
if (ret != 0)
{
throw std::runtime_error("Cannot create pipe");
}
}