Thanks for your feedback here. Changing the reconnection logic would be a long term consideration as even our Zoom Client App/s use the same logic.
If there is anything we can do to help you better detect a disconnection so you can remove or hide the user while they are in the “ghost” state, please let us know.