Want to create a customize zoom app in React JS

Manas · March 10, 2025, 10:31am

I want to integrate the Zoom app into my React website, but I need to customize it to display the real-time heartbeat of all participants in an ongoing Zoom meeting. Specifically, I want to detect each participant’s face and display their heartbeat on their face during the meeting. However, I’ve read that I can’t access the raw video feed from Zoom through the Web SDK for this type of task.

Can anyone help me figure out how to handle this issue?

Please help me out.

Thank you!

alexm · March 11, 2025, 3:27am

This sounds like a really interesting concept.
I have previously used face-api.js to determine if a guest is too close/far from their camera and if they need to move left or right, or move higher or lower, to be perfectly centered for a call; I had to make some custom CSS stuff to tie into that and output to canvas over the webrtc embed. If you knew in advance who the participants were and had good photos for face detection to work, it’d probably be pretty easy to correlate the heartbeat data to the right person and size/position the overlay properly.

alexm · March 11, 2025, 3:31am

Following up: If you’d need that overlay to sent back to all the participants, it’d take some looping back of your app’s render back into your camera input. Not impossible but not doable purely within react (I’m thinking your app would be open and being screen captured by something like OBS, then your Zoom would use OBS as the camera; participants would have to pin you for it to work right, probably).

vic.yang · March 14, 2025, 8:25am

Hey @Manas

Thanks for your feedback.

We have an official sample app built with React on GitHub

Regarding the raw data requirement, we currently provide APIs to manipulate the captured video stream. Support for raw data in the receiving stream will be available in a future release.

Thanks
Vic

Manas · March 17, 2025, 7:57am

Thanks for the response, Vic! I’m currently using the same video SDK you suggested. Could you please clarify which API you’re referring to for manipulating the captured video stream? Also, I would like to know if it’s possible to send the processed video (with the heartbeat display on participants’ faces) back to Zoom for playback during the meeting.

If this is feasible, could you guide me on how to achieve this and which specific APIs I would need to use?

Thank you!

vic.yang · March 17, 2025, 8:12am

Hi @Manas

We have a section that introduces how to use this function, and in the sample app, we also provide the code implementation.

github.com/zoom/videosdk-web-sample

src/processor/watermark-processor.ts

master

/* eslint-disable @typescript-eslint/explicit-member-accessibility */
class WatermarkProcessor extends VideoProcessor {
  private context: OffscreenCanvasRenderingContext2D | null = null;
  private watermarkImage: ImageBitmap | null = null;

  constructor(port: MessagePort, options?: any) {
    super(port, options);

    port.addEventListener('message', (e) => {
      if (e.data.cmd === 'update_watermark_image') {
        this.updateWatermarkImage(e.data.data);
      }
    });
  }

  async processFrame(input: VideoFrame, output: OffscreenCanvas) {
    this.renderFrame(input, output);
    return true;
  }

This file has been truncated. show original

Thanks
Vic

system · April 16, 2025, 6:12pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Create a connection between a React web app and zoom API and Webhooks	5	3938	March 22, 2021
React component from web client SDK Web	6	2297	August 21, 2020
How to integrate zoom with reactjs and nodejs Web	7	21345	July 28, 2020
Integrating with a custom application Web	2	586	November 5, 2020
Displaying dynamic data on the video stream Web	2	493	June 17, 2021

Want to create a customize zoom app in React JS

Related topics