Send raw video in meeting bot

Meeting SDK
1

I’m trying to send a video to my bot’s image after he joins a meeting.

Created a yuv file from a single image file.
Here is my implementation to send the frames:

void PlayVideoFileToVirtualCamera(IZoomSDKVideoSender* video_sender, const std::string& video_source) {
    int frame_rate = 25;

    int frame_duration_ms = 1000 / frame_rate;
    std::ifstream file(video_source.c_str(), std::ios::binary);
    if (!file.is_open()) {
        std::cout << "Failed to open file: " << video_source << std::endl;
        return;
    }

    // Calculate the frame size for I420 format
    std::cout << "width: " << width << " and height: " << height << " of the sending video" << std::endl;
    int frameSize = width * height * 3 / 2;

    // Allocate buffer for one frame
    char *frameBuffer = new char[frameSize];

    std::cout << "Sending the image " << std::endl;

    auto start_time = std::chrono::steady_clock::now();
    file.read(frameBuffer, frameSize);

    while (true) {
        video_sender->sendVideoFrame(frameBuffer, width, height, frameSize, 0);
        std::this_thread::sleep_for(std::chrono::milliseconds(frame_duration_ms));
    }

    std::cout << "finished sending image" << std::endl;
    // Clean up
    delete[] frameBuffer;
    file.close();
}

It sometimes works and sometimes not. totally randomly.
When I’m using Ipad for a meeting it shows the image.
When I’m using the mac zoom client it doesn’t present the image.

After some further debugging.
this call returns no errors:

SDKError err = p_videoSourceHelper->setExternalVideoSource(virtual_camera_video_source);

This code executed as well:

void ZoomSDKVideoSource::onInitialize(IZoomSDKVideoSender* sender, IList<VideoSourceCapability>* support_cap_list, VideoSourceCapability& suggest_cap)
{
    std::cout << "ZoomSDKVideoSource onInitialize waiting for turnOn chat command" << endl;
    video_sender_ = sender;
}

But for some reason, sometimes this method just not called:

void ZoomSDKVideoSource::onStartSend()

What can cause that?
(cc: @chunsiong.zoom )

Thanks.

2

@gofmannir

void ZoomSDKVideoSource::onStartSend() is triggered then the video is turned on (video unmuted)

3

@gofmannir the sample code listens for “turnOn” command in the chat, which in turn calls the method to send raw video.

4

@chunsiong.zoom
What is the turnOn command?

When I’m on mac zoom client, the support_cap_list length is 0.
When I’m on Ipad zoom client, the “onStartSend” called and the thread is strated. (Not always the sending frames works but this is a separate question), but never used the “turnOn” command, what is it ?

Thanks

5

@gofmannir,

Could you specify which Linux SDK sample you’re using? Since there are two different ones available, it would be helpful to know the specific SDK in order to better understand what might be happening.

6

Hi @donte.zoom , Started with the sample code: GitHub - zoom/meetingsdk-linux-raw-recording-sample

7

@gofmannir do a search on the code, “turnOn” is a string which the sample code listens for in the meeting chat.

8

Hi @chunsiong.zoom ,The only place in the sample in the print

std::cout << "ZoomSDKVideoSource onInitialize waiting for turnOn chat command" << endl;

There is no logic in the sample related waiting to “turnOn” string in the chat, are you sure?

9

@chunsiong.zoom
After some further debugging, I’ve got an error got 11.

This is my code:

void CheckAndStartRawSendingBot(){
	ZoomSDKVideoSource* virtual_camera_video_source = new ZoomSDKVideoSource(DEFAULT_VIDEO_SOURCE);
	IZoomSDKVideoSourceHelper* p_videoSourceHelper = GetRawdataVideoSourceHelper();

	if (p_videoSourceHelper) {
		SDKError err = p_videoSourceHelper->setExternalVideoSource(virtual_camera_video_source);

		if (err != SDKERR_SUCCESS) {
			printf("attemptToStartRawVideoSending(): Failed to set external video source, error code: %d\n", err);
		}
		else {
			printf("attemptToStartRawVideoSending(): Success \n");
			IMeetingVideoController* meetingController = m_pMeetingService->GetMeetingVideoController();
			
			SDKError err = meetingController->UnmuteVideo();
			if (err != SDKERR_SUCCESS) {
				printf(" meetingController->UnmuteVideo(): Failed, error code: %d\n", err);
			}else{
				printf("meetingController->UnmuteVideo() success ");
			}
		}
	}
	else {
		printf("attemptToStartRawVideoSending(): Failed to get video source helper\n");
	}
}

Got this error after calling unmuteVideo(),

meetingController->UnmuteVideo(): Failed, error code: 11

Which is SDKERR_VIDEO_NOTREADY right?

What can cause that?

10

@gofmannir

There are 2 methods which have already been created.
turnOnSendVideoandAudio and turnOffSendVideoandAudio.

That comment was from test, but the chat event listener wasn’t shipped in the current sample.

  1. The SDK will listen for the command
  • “turnOn” and call turnOnSendVideoandAudio
  • “turnOff” and call turnOffSendVideoandAudio

I’ve used chat message as an event trigger, which in turn calls these 2 methods

The reason being, you need to call these from the main thread, and using the chat message as a trigger would work.

11

Understood, so as I showed in the previous comment, I craeted a CheckAndStartRawSendingBot method, called in the main thread, but unmuteVideo returns error code 11. why is that?

12

@gofmannir because the video is not ready yet.

Sending raw video is tricky due to dependencies. I’m using chat to trigger the send as this will reduce the race condition.

You do not have to use chat, and can use other different triggers, such as on user join.

14

@chunsiong.zoom So there is a way to wait until the service is ready? or set event

16

@chunsiong.zoom Fixed by creating a thread that tries to unmutVideo every 2 seconds until there is nor error