Zoom Bot Streaming Video

Meeting SDK
1

I am working on a project where I want to stream a bot’s video/avatar to a meeting. I know I can create a Zoom bot, but I’m having trouble understanding which SDKs and APIs I can use (or if they even exist) to stream video TO the zoom meeting. Any help is appreciated!

2

@engineeredinnovation, happy to break down how you could build a meeting bot and stream audio/video to the bot avatar.

  1. Spin up a server. We recommend AWS, GCP, or Digital Ocean.
  2. Use either the Windows or Mac Zoom SDK to launch an instance of the Zoom client.
  3. Once you have the Zoom SDK launched, and use the video output functionality to stream video into the meeting.
  4. Once you have one instance of this working, you’ll need to scale this across several servers if you want to run multiple bots simultaneously, which is required to have bots for multiple meetings.

Another option is Recall.ai. It’s a simple 3rd party API that lets you use meeting bots to get the raw audio/video from meetings + output video/audio without you needing to spend months to build, scale and maintain these bots.

Let me know if you have any questions!

1 Like
3

@engineeredinnovation ,

You will need to run your bot on macos, windows or linux.

  • Create your own avatar using your own libraries or 3rd party library
  • Stream the avatar’s video using raw video send capabilities of the SDK (macos, windows or linux)

This send capability is something like a virtual camera, where you can stream in your own video source instead of getting it from a webcam

4

Are there any examples of streaming video using the raw video send capabilities for the Linux Meeting SDK?

5

Welcome for the dev forum @chloe3!
Here is an example of the Linux SDK, but I don’t think it showcases raw video send capabilities: GitHub - zoom/meetingsdk-linux-raw-recording-sample (courtesy of @chunsiong.zoom)

Another option is Recall.ai. It’s a simple 3rd party API that lets you use meeting bots to send raw audio/video into meeting meetings + capture video/audio without you needing to spend months to build, scale and maintain these bots.
Let me know if you have any questions!

6

@chloe3 what is the use case for sending raw video?
I’m checking if there is demand for such samples

7

@chunsiong.zoom similar to the use case above, I have a pre-recorded video of a bot that I would like to join the meeting with a greeting. Is it possible to have the bot join and have the video play as the bots camera or have the bot share its screen to play that video?

8

@chloe3 ,

have the bot join and have the video play as the bots camera

This scenario is definitely possible.

You can use the IZoomSDKVideoSource interface to do so.
It is similar to specifying your own webcam, but instead of a device, you are playing the video via a file.

Do note that sending audio is a separate thing, and requires the use of a virtual microphone

	//SendVideoRawData
	if (isVideo) {
        std::string DEFAULT_VIDEO_SOURCE = "Big_Buck_Bunny_720_10s_10MB.mp4";
		ZoomSDKVideoSource* virtual_camera_video_source = new ZoomSDKVideoSource(DEFAULT_VIDEO_SOURCE);
		IZoomSDKVideoSourceHelper* p_videoSourceHelper = GetRawdataVideoSourceHelper();

		if (p_videoSourceHelper) {
			SDKError err = p_videoSourceHelper->setExternalVideoSource(virtual_camera_video_source);



			if (err != SDKERR_SUCCESS) {
				printf("attemptToStartRawVideoSending(): Failed to set external video source, error code: %d\n", err);
			}
			else {
				printf("attemptToStartRawVideoSending(): Success \n");
				IMeetingVideoController* meetingController = m_pMeetingService->GetMeetingVideoController();
				meetingController->UnmuteVideo();

			}
		}
		else {
			printf("attemptToStartRawVideoSending(): Failed to get video source helper\n");
		}
	}

	//SendAudioRawData
	if (isAudio) {
        std::string DEFAULT_AUDIO_SOURCE = "Big_Buck_Bunny.wav";
		ZoomSDKVirtualAudioMicEvent* audio_source = new ZoomSDKVirtualAudioMicEvent(DEFAULT_AUDIO_SOURCE);
		IZoomSDKAudioRawDataHelper* audioHelper = GetAudioRawdataHelper();
		if (audioHelper) {
			SDKError err = audioHelper->setExternalAudioSource(audio_source);
		}
	}
9

Thanks @chunsiong.zoom!

I have added in this code, but when trying it out I keep getting the below error - I was wondering if there is a particular format or size the video needs to be?

onStartSend
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (873) open OpenCV | GStreamer warning: Error opening bin: unexpected reference "test-video.mp4" - ignoring
[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
ERROR! Unable to open camera
10

@chloe3 this seems like an openCV issue.

To send video, you need to ensure it is in YUV420 format

11 
    const client = ZoomMtgEmbedded.createClient()
    let meetingSDKElement = document.getElementById('meetingSDKElement')
    client.init({ zoomAppRoot: meetingSDKElement!, language: 'en-US' })
    await client.join({
        sdkKey: sdkKey,
        signature: generateSignature(sdkKey, clientPassword, meetingNumber, "1"),
        meetingNumber: meetingNumber,
        password: meetingPassword,
        userName: userName
    });

I am currently utilizing this for connecting with a client, but now I desire to connect with a bot and configure the bot’s video. How can I accomplish this?

12

@praveen.i please open a seperate thread and tag me