# EIS functionality

Source: https://forum.modalai.com/topic/4843/eis-functionality
Category: VOXL SDK and Software (https://forum.modalai.com/category/47/voxl-sdk-and-software)
Tags: camera
Posted: 2025-10-29 16:01:25 UTC by SKA
Replies: 13 · Views: 1421

## SKA · 2025-10-29 16:01:25 UTC

Hi! I'm noticing some difference in functionality when I switch ```eis_mode``` from ```horizon-level``` to ```full-follow```. 

I built ```voxl-camera-server``` off the ```eis``` branch and followed the instructions. everything seems to be working correctly including stabilization other than the final output rotation that I am looking for.

My ```imu_apps``` to ```hires``` extrinsics are set up correctly and I want to apply this additional transform to the output:

     {
            "parent": "hires",
            "child":  "hires_eis",
            "T_child_wrt_parent": [0.066, 0.009, -0.012],
            "RPY_parent_to_child":    [0, 0, -90]
        }

I'm noticing that if I set ```"eis_mode":	"full-follow"``` the transform isnt applied but if i set ```"eis_mode":	"horizon-level"``` it rotates the output window as expected. Here's what I see on ```voxl-portal``` when I switch ```eis_mode``` and leave the extrinsics file untouched.

```"eis_mode":	"full-follow"```:
![146f1c37-7919-49a7-9e2a-b4581cc90629-image.png](https://forum.modalai.com/assets/uploads/files/1761753522656-146f1c37-7919-49a7-9e2a-b4581cc90629-image.png) 

```"eis_mode":	"horizon-level"```:
![105fa5c2-0d7c-4d91-a8f7-bad7825cb8e8-image.png](https://forum.modalai.com/assets/uploads/files/1761753512631-105fa5c2-0d7c-4d91-a8f7-bad7825cb8e8-image.png) 

Hoping to find a way to get the output rotation working in ```full-follow``` mode. Thanks!

## Reply by Alex Kushleyev (ModalAI staff) · 2025-10-29 17:37:36 UTC

@SKA , I thought i fixed that issue a while ago, let me double check. Are you using `voxl-camera-server` from `eis` branch or an official release from an SDK?

Alex

## Reply by SKA · 2025-10-29 17:40:18 UTC (in reply to Alex Kushleyev)

@Alex-Kushleyev I'm building from the ```eis``` branch not the official release. Thanks!

(Note: The images I included in my post are labeled backwards. first image is ```"eis_mode": "horizon-level"``` and the second image is  ```"eis_mode": "full-follow"```)

## Reply by Alex Kushleyev (ModalAI staff) · 2025-10-29 19:04:55 UTC (in reply to SKA)

@SKA , OK, so what you are saying is that in `full-follow` mode, the rotation parameter in the extrinsics `hires` -> `hires_eis` for the -90 degree rotation is not respected, correct?

## Reply by SKA · 2025-10-29 19:31:07 UTC (in reply to Alex Kushleyev)

@Alex-Kushleyev yes, that's correct!

## Reply by Alex Kushleyev (ModalAI staff) · 2025-10-30 03:53:53 UTC (in reply to SKA)

@SKA ,

In your case, the issue should be solved by commenting out one line in https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/eis/src/misp.cpp :

```
       //TODO: FIXME
       //in roll following mode, the roll is not stabilized to the horizon, so it does not need to be flipped
       //need to resolve this in a better way because Rout is meant for another purpose
       if (!follow_roll)
           rc_matrix_right_multiply_inplace(&eis_ctx.H.m, eis_ctx.Rout.m);    //rotate the image according to desired output rotation (nominally identity)
```

Specifically, comment out the `if` statement : `if (!follow_roll)` , so that the `Rout` matrix is always applied to the full transform H.

The issue is that I tried to solve another use case by using this parameter (when the voxl2 / IMU is flipped upside down while camera is right side up). I just tested and it appears to be working.

I will figure out a better way of handling the upside down IMU in this case so that the output transform can be used correctly, but for now you can just comment out that `if` statement.

Can you please let me know if that fix worked for you?

Alex

## Reply by Alex Kushleyev (ModalAI staff) · 2025-11-10 20:54:08 UTC (in reply to Alex Kushleyev)

Hi @SKA ,

I believe i fixed this issue with `full-follow` mode in the camera server. Now EIS has a body->imu transform, which is read from the extrinsics file and is used to properly rotate the image, instead of hi-jacking the cam->cam_eis transform.

So now your cam->cam_eis transform should work. Please make sure the imu_apps->cam transform is correct, if your camera is rotated 90 degrees.

Please check the commits from Nov 10 : https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/commits/dev

Alex

## Reply by SKA · 2025-12-05 17:56:27 UTC (in reply to Alex Kushleyev)

@Alex-Kushleyev Thanks Alex! I tried testing this out, but it looks like it depends on a newer version of libmodal-pipe. 

```
voxl2-mini (M0104):/data/voxl-suite-offline-packages$ dpkg -i --force downgrade --force-depends voxl-camera-server_2.2.16_arm64.deb
(Reading database ... 102703 files and directories currently installed.)
Preparing to unpack voxl-camera-server_2.2.16_arm64.deb ...
Unpacking voxl-camera-server (2.2.16) over (2.2.4) ...
dpkg: voxl-camera-server: dependency problems, but configuring anyway as you requested:
 voxl-camera-server depends on libmodal-pipe (>= 2.14.0); however:
  Version of libmodal-pipe on system is 2.13.2.

Setting up voxl-camera-server (2.2.16) ...
voxl2-mini (M0104):/data/voxl-suite-offline-packages$ voxl-camera-server
voxl-camera-server: symbol lookup error: voxl-camera-server: undefined symbol: pipe_validate_cpu_stats2_t
```
would you recommend upgrading the SDK to 1.6.0, or is there another way to get this working? I would prefer not to upgrade the SDK.

Thanks for your help!

## Reply by Alex Kushleyev (ModalAI staff) · 2025-12-05 18:20:42 UTC (in reply to SKA)

@SKA, you should be able to update libmodal-pipe to latest package in the dev package repository : http://voxl-packages.modalai.com/dists/qrb5165/dev/binary-arm64//

## Reply by SKA · 2025-12-05 20:04:15 UTC (in reply to Alex Kushleyev)

@Alex-Kushleyev Thanks! After upgrading libmodal-pipe (which also requires a voxl-cpu-manager upgrade), I was able to install and run voxl-camera server, as well as the latest version of voxl-portal from the eis-integration branch. 

I followed the instructions [here](https://gitlab.com/voxl-public/voxl-sdk/services/voxl-camera-server/-/blob/eis/voxl2-eis.md?ref_type=heads), and set up eis. 

when I run voxl-camera-server, I'm seeing this output and I cannot see any camera stream on voxl-portal. Let me know if you recognize this issue and know how to fix it. Thanks!

![b3b91a50-2d61-41b0-909b-a384a213e0db-image.png](https://forum.modalai.com/assets/uploads/files/1764965024004-b3b91a50-2d61-41b0-909b-a384a213e0db-image.png) 

![de8b1ee2-3316-4047-b736-1140467a0230-image.png](https://forum.modalai.com/assets/uploads/files/1764965049325-de8b1ee2-3316-4047-b736-1140467a0230-image.png)

## Reply by SKA · 2025-12-05 21:09:57 UTC (in reply to SKA)

After upgrading to sdk 1.6.0 things are working well and the transform is incorporated correctly. Thanks for your help! If you know how I can get it working on sdk 1.5.0 I'd still be interested in knowing how.

## Reply by Alex Kushleyev (ModalAI staff) · 2025-12-05 22:20:23 UTC (in reply to SKA)

@SKA , that result / buffer error (i think) is showing up because the camera pipeline selects incorrect camera mode / resolution, perhaps you missed one of the steps in EIS instructions, either using the correct camera drivers or updating the override txt file.

Also, the latest docs for EIS are here, the old .md file is probably outdated : https://docs.modalai.com/camera-video/electronic-image-stabilization/

Alex

## Reply by SKA · 2025-12-05 22:27:51 UTC (in reply to Alex Kushleyev)

@Alex-Kushleyev I verified that I used the correct drivers and had already updated ```maxRAWSizes``` on ```/vendor/etc/camera/camxoverridesettings.txt```. I just went through the updated EIS docs and I am still seeing the same errors. Let me know if there's anything else I can check!

## Reply by Alex Kushleyev (ModalAI staff) · 2025-12-06 00:23:23 UTC (in reply to SKA)

@SKA ,

Does this happen right away after you start `voxl-camera-server` ?

What raw "preview" resolution / fps are you requesting in camera server config?

Also, can you post output of dmesg (everything after you start voxl-camera-server)

Alex