Virtual Background not working in Preview Camera

[shoaib]

Description
We want to show virtual background in preview camera.
And for that we found one API → previewVirtualBackground but it is expecting HTMLCanvasElement as param, and for preview camera we use HTMLVideoElement as mentioned here.

Is there any way to do it or any other API which we can use for this case?

[rehema.zoom (Rehema)]

Hi @shoaib ,

Thanks for your question. What browser are you seeing this error on? The reason may be that a virtual background requires SharedArrayBuffer be enabled (on Chrome or Edge), and SharedArrayBuffer calls for a ‘canvasElement’, rather than a ‘videoElement’. Can you try rendering with a canvas element and see if it works?

I definitely see how the documentation can be a bit misleading and will request a change to the team.

Thanks,
Rehema

[shoaib]

hey @rehema.zoom ,
There is no error, the issue is that I need to show Virtual Background in Local Preview Video for that I found this API: previewVirtualBackground but it requires a canvasElement which we don’t have in Local Preview Component.

Preview camera renders video on HTMLVideoElement only as also mentioned here.

You can see in above mentioned code, to start a preview video we pass a HTMLVideoElement in localVideoTrack.start(), So there is no canvasElement element that we can pass in previewVirtualBackground API.

I’ve tried the Preview in demo app as well, there also Virtual Background is not working in Local Preview.

[rehema.zoom (Rehema)]

Got it, thank you for the clarification. Just to rule out some other issues, what browser are you using for your app and testing the demo app in? I’m doing some continued research on the API and will reach back out when I have an update.

Thanks,
Rehema

[shoaib]

hey @rehema.zoom ,
I’ve tried it in both Chrome and Firefox.

[rehema.zoom (Rehema)]

Hi @shoaib ,

Thanks for the follow-up. Previewing a virtual background is actually not available for the pre-session preview of your video; it’s only available once you’re in a session. The API mentioned in this thread is for in-session use only; my apologies for the confusion. I hope this provides some clarity! I will work on making sure the documentation more clearly states this.

Thanks,
Rehema

[shoaib]

Hey @rehema.zoom ,
Got it, Thanks for clarifying.
Is there any plan or ETA on addition of an API for this use-case,
because user would want to first check Virtual Background pre-session as well before actually applying it, right? → It is a highly probable expectation from the customer’s perspective.

[shoaib]

hey @rehema.zoom, any update on the above query?

[rehema.zoom (Rehema)]

Hi @shoaib ,

Thanks for your feedback.I checked with the team and, at this time, there is no plan to add virtual background to the video preview. This is due to significant differences in the tech stack used for the preview component (for the web platform).

Thanks,
Rehema