Can't start preview localVideoTrack without background with Canvas

Video SDK
1

I want to create a preview settings screen before the user joins the session.
Here, the user can preview a virtual background, among other things.

I recently learned that we need to use <canvas> and not <video>, like the docs incorrectly mention.

However, when I run track.start(element); I get the error Cannot start video with virtual background. This can’t be right, because I am not providing any virtual background in the start call, as you can see.

Furthermore, this message is weird, because I’m finding exactly the opposite thing. I can start the video preview with a virtual background, but I can’t do it without one (because of this error).

The sample app does a very hacky thing which is to render both a video and canvas, and then hide one or the other depending on if the user needs a virtual background or not.
This feels like a workaround and an implementation detail that should be abstracted away from SDK users.

Another thing that I found is that, with <video> we can create multiple video tracks. With <canvas>, you can only have a single video track and have to re-use it.
So, let’s say you have a preview dialog. You need to account for it and make sure to reuse the local video track instance and not throw it away and recreate it afterwards.

Canvas in video preview seems buggy at the moment.

2 Likes
2

Hey @miguel2

Thanks for your feedback.

I apologize for the inconvenience you’ve experienced with the Video SDK Web API.

In the current preview video without a virtual background, we use the standard WebRTC implementation. However, when switching to virtual background, our current approach involves capturing frames through WebAssembly and then rendering them onto a canvas. This aligns with what you mentioned above about creating both canvas and video tags in the sample app.

We are working on further optimizations to unify the calling approach in the future release.

Thanks
Vic

3

@vic.yang related question: how do I create video SDK credentials?

Sign into your Video SDK account. Go to the Zoom App Marketplace, hover over Develop , and click Build Video SDK .

However, there is no such option as “Build Video SDK” in the Develop menu:
image

Edit: for some reason, it appears now. But even though I’m using the correct key and secret, I get This account does not exist or does not belong to you.

This is what I see in the page:

screenshot-marketplace.zoom.us-2024.01.12-16_25_25
screenshot-marketplace.zoom.us-2024.01.12-16_25_251050×2042 228 KB

There is no “SDK Credentials” section. The secret and token visible on the screenshot seem to be for event subscriptions, not for the SDK.

4

@miguel2 ,

From the screenshot you have shared, the signed in account is a meeting SDK account.
You will need to sign up for a Video SDK account with another email address.

5

That is correct, yes. I was able to create a Video SDK account.

The problems with video preview remain, though.
Would be great to have a better approach for preview.

6

I am having a similar issue with Video Back ground in Video Preview / Local Video Track.
I am trying to add blur on my canvas html element when starting a track, but the video does not start.

if I change to a Video Html element, it works the video but no blur.

<canvas id=“preview-camera-video” width=“1280” height=“720” playsinline #videoElement>

const videoSettings: PreviewVideoBackground = {
  imageUrl: 'blur',
  cropped: false
};

this.localVideoTrack = ZoomVideo.createLocalVideoTrack(cameraId);

await this.localVideoTrack.start(videoDOMElement, videoSettings);

7

videoDOMElement was not being casted to HTMLCanvasElement, I made the changes below:

await this.localVideoTrack.start(videoDOMElement as HTMLCanvasElement);

And now I get the same error as the OP:

Error Error: Cannot start video with virtual background
at index.umd.js:1:662106
at Generator.next ()
at asyncGeneratorStep (asyncToGenerator.js:3:1)
at _next (asyncToGenerator.js:22:1)
at asyncToGenerator.js:27:1
at new ZoneAwarePromise (zone.js:1411:21)
at asyncToGenerator.js:19:1
at GD.start (index.umd.js:1:662306)

If I provide :

{ imageUrl: ‘blur’ }

I don’t get the error, but the video does not start, nor I get a video with blur background :frowning:

8

It looks like we need to initialize zoom client before using preview, this was not clear in the documentation.

async intializeClient(): Promise {
await this.ngZone.runOutsideAngular(async () => {
if (!this.zoomVideo) {
this.zoomVideo = ZoomVideo.createClient();
await this.zoomVideo.init(‘en-US’, ‘Global’, { patchJsMedia: true });
}
});
}

9

Hey @victor.mello

Thanks for your feedback.

I apologize, this was an oversight in our documentation. Using virtual background needs to be done after client.init method .

Thanks
Vic

1 Like
10

Seems weird that we can’t start a preview with canvas without a virtual background.
One would think that not using a virtual background would be simpler!

2 Likes
11

Amazing information. learned a lot

14

Hey @victor.mello

I found a new issue …

I understand the ZoomVideo.createLocalVideoTrack() should be used after client.init and it works for me…

We need to implement a ‘blur’ button that enables users to switch their virtual background during the ‘pre-session’ phase, which utilizes the localVideoTrack.

The feature works whether it’s set to blur by default or not; however, we are unable to toggle it on and off.

When toggling, I first stop the localVideoTrack using localVideoTrack.stop(), then I attempt to restart it with localVideoTrack.start(), specifying either an image blur or undefined. Despite this, it still doesn’t permit changes.

It works in Chrome, but not in Firefox and Safari.

The javascript console log shows looping error… “Unhandled Promise Rejection: InvalidStateError: The object is in an invalid state.”

any clue ?

Thanks !