I’ve integrated the Zoom Video SDK React UI Toolkit into my project and enabled the annotation feature.
Drawing and pointer annotations work as expected, but text annotation (typing text on the shared screen) is not showing up or available in the toolbar.
I would like to confirm:
Is text annotation currently supported in the Video SDK (Web / React UI Toolkit)?
If not, is there any plan to include text annotation support in future updates?
Are there any suggested workarounds to implement text annotation manually?
Browser Console Error
No specific console errors. The annotation toolbar loads normally, but text annotation tools are missing.
Hi @krishnan At the moment, text annotation is not a supported for the Web Video SDK or UIToolkit. It is possible to implement this feature using Web Video SDK Share processors and drawing the desired text directly onto the canvas frame: Video SDK - web - Raw data - share
Works perfectly: Full desktop screen sharing - annotations update in real-time
Problem: Single app window screen sharing - annotations only appear when I hover over the shared app
When sharing a single application window, the processFrame() method in my ShareProcessor only triggers when I interact with (hover/click) the shared application window.
This appears to be because the OS only provides new video frames to the capture system when the application window actually repaints/updates. Static windows don’t generate paint events, so no new frames are fed to the ShareProcessor pipeline.
So, my questions:
UI Toolkit Implementation: Annotations perfectly working for single app screenshare in ReactJS UI toolkit. How does the Web UI Toolkit handle annotations for single app screen sharing?
Alternative Approaches: Would a chat-based annotation system (sending x,y coordinates and text via Zoom’s messaging system and rendering as DOM overlays) be a viable alternative?
Recording Considerations: If we implement overlay-based annotations (not baked into the video stream), How can we include these annotations in recording session?
Hi @krishnan happy to see you checked out the Share Processor! You are correct, processFrame will only send new frames when repaints or movement on the page is detected. For the annotation use case, the way I would implement this is starting and stopping the processor when the annotation needs to occur based on post message events. By starting the processor, it is guaranteed to run at least once. With this in mind, you could draw the annotation and stop the processor. When another annotation is needed, you start the processor again. You could save the x,y coordinates of these annotations to save the previous annotations.
To answer your questions:
UI Toolkit Implementation: Annotations perfectly working for single app screenshare in ReactJS UI toolkit. How does the Web UI Toolkit handle annotations for single app screen sharing
The UItoolkit implements the annotation included in the share feature of the Video SDK. This includes various drawing tools that the SDK itself implements
Alternative Approaches: Would a chat-based annotation system (sending x,y coordinates and text via Zoom’s messaging system and rendering as DOM overlays) be a viable alternative
I can see a use case where you can use the command channel feature for this.
Recording Considerations: If we implement overlay-based annotations (not baked into the video stream), How can we include these annotations in recording session?
While this is possible on the client device, the caveat is only cloud recording is supported with the Web Video SDK. So only media data captured within the mediaStream will be recorded
For the annotation use case, the way I would implement this is starting and stopping the processor when the annotation needs to occur based on post message events.
Does that mean my issue (single app screen-share annotation problem) can be fixed by implementing processor start/stop?
If not, which method can be used to add text annotation directly to the stream from canvas ?
