@Gary-Holmgren , we have not tried the mask with 1280x800 resolution.
Can you please try using the mask of size 160x120 and make it half white / half black for sanity testing. It is possible that the mask size has to be of these specific dimensions (and it would get resized to match the aspect ratio of the actual image, not sure).
Also, you should enable the debug prints in QVIO, as mentioned here https://forum.modalai.com/topic/3459/masking-qvio/6 and there is another potential "gotcha", the mask path is relative to working directory where qvio server starts.. but after you enable the qvio server debug prints, you will see the path from which the mask file is being loaded, as you can see from screen shots from the other post.
Unfortunately, the underlying QVIO source code is not available, so I cant check the exact behavior of the masking feature.
Alex