I was thinking of something simple, eg:
Use tracking cam qvio until reported quality drops below a particular value (near blowup). If it does, switch to stereo qvio feed as long as the position + velocity were recently in agreement within some threshold and its reported quality is sufficient. If not, switch to manual mode and call an adult. Once the main qvio recovers, store the position offset (for future switchovers) and switch back.
I'm sure proper fusion will obtain a smoother and more robust result, but I'd just like something that can handle a blowup without crashing.