How to get realtime meeting video and audio data

[sahar.b1 (Sahar)]

Hello everyone! Happy to join

So, my use case -
I need to get the meeting video and audio (raw) in real-time, right to my back-end, to do my logic and display meeting insights in the meeting chat in real time

Is there any other way to get real-time data other than using the Livestream Meeting feature?
API? Webhook?

Thanks a lot,

[amanda-recallai]

@sahar.b1 , Here are 3 methods people typically use to get the real-time raw video and audio from Zoom.

1. Use the Zoom RTMP live-streaming API

Pros:

• Doesn’t require any 3rd party services
• Lighter weight than building and running a Zoom bot

Cons:

• Needs to initiated on a per-meeting basis
• You need to set up an RTMP server to receive the data, which requires engineering effort to deploy, scale, and monitor
• No speaker separation

2. Build a Zoom bot

Pros:

• Can get the separate audio and video streams per participant for perfect diarization / speaker labels

Cons:

• It is very heavy-weight as you would need to spin up multiple servers to run the Zoom client for the bot
• Running infrastructure for Zoom bot costs more than live streaming.
• You need to encode the raw video and audio yourself

3. Use the Recall.ai API

It’s a unified API that lets you send meeting bots to video conferencing platforms to capture the audio, video and transcription in real-time.

Pros:

• Zoom has an official Meeting Bot Starter Kit they created with Recall.ai
• Gets the real-time video and real-time audio data for you - you just need to call an API endpoint.
• You can send messages in the Zoom meeting chat via their APIs as well.
• Works on any Zoom plan (including Free)
• Gets speaker diarization / speaker labels
• Works agnostic of meeting platform

Cons:

• It’s another 3rd party service in your stack

Let me know if you have any questions!

[sahar.b1 (Sahar)]

Many thanks @amanda-recallai !
Your helpful answer led me to either #2 or #3 options.

Are there any latency details for both options?