Selection of Video vs Canvas rendering

Video SDK
1

Video SDK Type and Version
Video SDK web, 2.2.12

Description
Hi all - I’m hoping to understand the internal selection process that occurs in VideoSDK web for whether a canvas vs multiple video elements should be used. For the same webpage, I get different results depending on if I’m in Chrome (canvas) vs Safari (video elements).

Additionally, our implementation makes use of a shared source code via an internal NPM package that is docked in two different host apps. The canvas render in Chrome indicated previously is on a next.js app. The same source code on a Vite SPA uses multiple video renders in Chrome.

This is causing some issues with unpredictability in our styles, and WebGL issues (I previously had too many video-player-containers) that only materialized when canvas was used as a render and we got above a certain number of concurrent videos. This was not detectable in the video render strategy.

We are using the attachVideo API in our app.

mediaStream
          .attachVideo(
            userId,
            VideoQuality.Video_720P,
            videoPlayerRef.current ?? undefined
          )
          .then((userVideo) => {
            if (!("errorCode" in userVideo) && !videoPlayerRef.current)
              videoPlayerRef.current =
                document
                  .querySelector(`#stream-container-${userId}`)
                  ?.appendChild<VideoPlayer>(userVideo) ?? null;
          });
  • What specifically determines which render strategy will be used?
  • Is there any control over which method is used to render the video? What specific consequences would there be of using video vs canvas in an environment where it would not have been selected?
  • Does the web SDK surface a check that we can use to detect how the video will be rendered for conditional styles, etc?

Thanks!

Pasted Graphic 1
Pasted Graphic 13066×504 110 KB

Pasted Graphic
Pasted Graphic1668×438 99.2 KB

2

Hey @apc

Thanks for your feedback.

We introduced the video-player web component to abstract away the differences between using a video element or a canvas underneath. If video-player is not sufficient for your needs or you prefer to use a specific underlying rendering element, note that this difference is determined by the video solution chosen by the Video SDK.

By default, the SDK automatically selects the most suitable solution based on the environment. However, we also provide the video_webrtc_mode field in the JWT to specify your preferred solution.

Thanks
Vic