Aha yes that was my original understanding. Yes knew you'd need a zoom level for it to actually look good/work effectively. Would probably have a lower bound of 1.2-1.3 likely from what I experimented with. TBD on dimensions.
Sorry, was confused, sounds good!
Awesome, sounds good Alex. That does indeed sound like it would meet our needs.
If EIS is enabled, How much smaller would the image be than 4040x3040? What is the max output resolution with EIS enabled?
Hi @Alex-Kushleyev,
Thanks for the reply.
The 4 AR0144 cameras would be used independently rather than as stereo pairs, so I don't believe strict synchronization is required for my use case. My main goal is to add a 4th AR0144 for an independent tracking/perception task.
As long as the 4th camera can be timestamped accurately and integrated alongside the other tracking cameras, I think that should be sufficient. The resistor swap seems a little tricky, so I might post again if I need help with that.
Thanks!
Will an AR0144 work on J8? Would probably need a Micro-coax Adapter? Not sure if its one output maps to Lower or Upper.
But couldn't find anything about if J8 would support AR0144. At the very least i was able to configure the camera-server with a AR0144 on both J8 Upper and Lower and it succeeded (although cannot test since i cannot connect the actual sensor).
Is this something that has been tested in the past?
For some context, am trying to see if we can run 4 AR0144s, 1 imx412, and 1 pmd-tof-liow2. Potentially some other sorting of these would work. (Such as putting the imx412 on J8 and another AR0144 on J6. Totally okay if it isn't an officially supported config haha can internally test.
current config with the starling breakout board looks like, looking to add one more AR0144
J6_LOWER_SENSOR="ar0144-fsin-combo"
J6_LOWER_NAME="arone"
J6_LOWER_ROTATE="true"
J6_LOWER_COMBO1_SENSOR="ar0144-fsin-combo"
J6_LOWER_COMBO1_NAME="artwo"
J6_LOWER_COMBO1_ROTATE="false"
J6_UPPER_SENSOR="imx412-fpv"
J6_UPPER_NAME="hires"
J6_UPPER_ROTATE="false"
J7_LOWER_SENSOR="ar0144-fsin"
J7_LOWER_NAME="arthree"
J7_LOWER_ROTATE="true"
J7_UPPER_SENSOR="pmd-tof-liow2"
J7_UPPER_NAME="tof"
J7_UPPER_ROTATE="false"
J8_UPPER_SENSOR=""
J8_UPPER_NAME=""
J8_UPPER_ROTATE="false"
Any help much appreciated, Thanks!
Best,
James
Hi Alex, apologies again!
Yes independent zoom is what we would be looking for. Awesome to hear you’ve been working on it. Yeah the three main use cases for us would be recording, streaming, and calibration (Makes it easier for an assembly process to not have to switch camera configs). So here is the minimum channel use case we were looking for
We currently use J6_UPPER_SENSOR: imx412-fpv
Channel 1 (Potential Recording Pipe 1 (Full Frame)):
Dimensions: 4040x3040
FPS: 30fps
Zoom: OFF/1x
EIS: OFF
EIS MODE: N/A
Channel 2: (Potential Recording Pipe 2 (Variable Zoom):
Dimensions: 4040x3040
FPS: 30fps
Zoom: Variable (1x-whatever, would be sync with streaming pipe. We would implement this logic)
EIS: ON
EIS MODE: Horizon (or full-follow)
Notes: Would likely leave it to our users whether to record full frame or be synced with streaming pipe. So would only ever need one of the recording pipes likely never both at once.
Channel 3: (Streaming Pipe):
Dimensions: 1280x800 (16:10 matches our tablet aspect ratio, not sure if there is something better to use here.)
FPS: 30fps
Zoom: Variable (1x-whatever)
EIS: On
EIS MODE: Horizon (or full-follow)
Channel 3 (Calibration Pipe):
Dimensions: 1010x760 (we’ve been calibrating at quarter resolution with our own internal calibration routine)
FPS: 30fps
Zoom: Off/1x (zero zoom full picture)
EIS: OFF
EIS MODE: N/A
Notes: Really only need the grayscale/normalized from this.
Anyways, happy to give it a try if you have available. Zero rush, currently juggling a few projects.
Best, James
Hi Alex,
Sorry has been awhile, keep having to step away and re-visit this.
First, seems like snapshot seems to be fixed in one of your latest dev releases, tried that out and seems to work so that’s awesome. Able to capture a full frame non-MISP image from the 412. That being said, although I tested out voxl-record-raw-image on the hires_misp_color stream, and was able to eventually convert it to a jpg, its not as usable if not in a standard image format by default.
In addition I had a few more clarifying questions.
1.
Seems like there’s no way to have the large hires pipe on at the same time as the MISP pipe? Meaning it’s not possible to get a recording of the full frame, no eis, at the same time the MISP pipe is being used? The reason why this might be useful is because although I could have a high-resolution MISP pipe for recording it would have the same zoom level as the other misp pipes as it seems that is shared parameter. I assume this is a system limitation and the voxl2 might not be able to handle having multiple 4k pipes and encoding them. Just want to verify this.
2.
I noticed there is a imx412 and imx412-misp driver. What is the difference between these two? MISP seems to work fine using the regular imx412 driver (and that’s what I’ve been using since https://docs.modalai.com/camera-video/electronic-image-stabilization/ didn’t make mention of the other one).
Is it just the same thing, different name or are there gains to be made switching to the dedicated MISP driver? What are those?
Let me know, Thanks!
P.S love to see the continued work being done by you all here on the camera server very cool!
Hi Alex,
To be greedy, ideally as many of those as possible. Ones that appeal most to us would be a
Then for format,
JPG probably makes the most sense but ofc, the more options the better haha.
Thanks for the detailed response Alex.
I think this addresses my most important concern, which was stretching of the misp pipes, I didn't realize that it was just cropping which is good news.
One more thing: Seems enabling Snapshots isn't compatible alongside MISP . Is that a fundamental limitation, or more of a how the stack is built thing? And is there currently any recommended way to still save a still image with MISP enabled? Or if not, a future implementation plan?
What I’m aiming for long term is high-res recording, lower-res streaming, and the option to grab a still in one configuration. With MISP ofc.
Best,
James
Working with EIS a bit more and was hoping for some clarification/advice on a couple things
First, want to clarify calibration for EIS. In the doc seems like it’s fine to calibrate at the full 4040x3040 (via disabling eis and setting misp_width and misp_height temporarily) or can alternatively calibrate at half-res at 2020x1520. Could I also hypothetically calibrate at 1010x760? We have our own calibration routine and the smaller the faster it can run.
I’m assuming as long as the aspect ratio is the same it should be fine? Furthermore, if I were to use the alternate resolution of 1996x1520 I should obviously do a calibration at that resolution? Or, keeping aspect ratio perhaps at 998x760?
Basically, just want to confirm that as long as a calibrate at some scale of whatever I set the preview frame to should be fine? And once I'm done calibrating I'm ok to then set misp_width and misp_height to my desired output resolution? This leads to my next clarification
I also wanted to clarify that EIS runs on whatever I set the preview frame to correct? As in that frame/resolution is what the algorithm runs on? misp_width and misp_height are just what gets outputted (as in its not like the algorithm is also getting run at that resolution)?
In that case what do you recommend eventually setting the output misp_width and misp_height to? Can they be anything? (Obviously something small enough to stream for example) I noticed in one example you chose 1280x720 but notably that is not the same aspect ratio as 4040x3040 or 1996x1520. Will stretching or warping occur? Should I pick something that is the same aspect ratio? Or does that not matter?
Any guidance much appreciated, thanks!