Zoom Meeting SDK Headless Bot in Linux - Audio starts ahead of time than video

[chriswuyiming (Chris)]

Meeting SDK Type and Version
Meeting SDKHeadlessLinux

Description
I am currently using your Zoom Meeting SDK Headless Bot on Linux to obtain audio and video streams from zoom meeting. I am experiencing an issue where the audio starts ahead of time compared to the video , it could potentially cause a synchronization issue. To ensure synchronization. I’ve implemented two named pipes to encode the audio and video streams at runtime, which has been working well up to this point. My next step is to stream the meeting through srt protocol to other device, my goal is to achieve low latency synchronization, eliminate high start up latency. I am looking for insights or suggestions on how to maintain tight synchronization with low start up latency, starting simultaneously to write video and audio data into pipes and low latency while streaming.

The bot requires over 6 seconds to write video data into file after the bot starts to write audio data into file.

Here is the output after 6 seconds

[chriswuyiming (Chris)]

@chunsiong.zoom @amanda-recallai

[amanda-recallai]

Hey @chriswuyiming ,

To ensure audio and video are always in sync, a common solution is to use something like GStreamer. It’s definitely possible to handle the offset yourself and process the media separately, but using GStreamer you can process the media in parallel and ensure it’s always in sync.

We actually run meeting bots at scale and leverage GStreamer ourselves, so feel free to let me know if you have any specific questions!

Alternatively, if you’re looking for a simple API to interface with the underlying audio/video streams, one other option would be to use Recall.ai for your meeting bots instead. It’s a simple 3rd party API that lets you use meeting bots to get raw audio/video from meetings without you needing to spend months to build, scale and maintain these bots.