drawImage in Camera Mode returning an error

wei.hu · July 19, 2022, 4:17pm

As Drawing images in Camera Mode Documentation in Zoom Apps, , I am trying to draw a image while in camera mode.

Steps to reproduce:

1.Start the rendering context in camera mode:

await zoomSdk.runRenderingContext({
    view: "camera"
})

Attempt to draw the image, eg:

const canvas = document.getElementById("canvas") as HTMLCanvasElement;
const ctx: CanvasRenderingContext2D = canvas.getContext("2d") as CanvasRenderingContext2D;
const img = new Image();

function draw(img: HTMLImageElement) {
    canvas.width = 1280;
    canvas.height = 720;

    // draw image
    ctx.drawImage(img, 0, 0);

    // get image data from canvas
    const imageData: ImageData = ctx.getImageData(0, 0, canvas.width, canvas.height);

    zoomSdk
        .drawImage({
            imageData,
            x: 0,
            y: 0,
            zIndex: 10
        })
        .then((ctx) => {
            console.log("drawImage returned", ctx);
        })
        .catch((e) => {
            console.log(e);
        });
}

img.onload = () => {
    draw(img);
};

img.src =
    "https://9bae041b1da7.jp.ngrok.io/assets/1@2x.jpg?0.0.14?w=164&h=164&fit=crop&auto=format";

This results in the following console error:

Is something missing from the documentation, or is there a bug with this feature?

Thanks!

narmadaravali · July 19, 2022, 11:08pm

Are you making sure if zoomSdk.runRenderingContext({ view: "camera" }) is called before you call await zoomSdk.drawImage({})?

drawImage needs camera context for it to draw image.

wei.hu · July 20, 2022, 12:41am

Yes, I’m sure the zoomSdk.runRenderingContext({ view: "camera" }) is called before I call await zoomSdk.drawImage({}), when I call the zoomSdk.runRenderingContext({ view: "camera" }), I got the error below,

Can you help me find out what is the reason? Thanks you!

wei.hu · July 20, 2022, 10:06am

Client version: 5.11.3 (8937)

My code is as below:

 const ctx = await zoomSdk.runRenderingContext({
    view: "camera"
 });

 console.log("runRenderingContext returned", ctx);

 zoomSdk
  .drawImage({
    imageData: new ImageData(100, 100),
        x: 0,
        y: 0,
        zIndex: 1
     }).then((ctx) => {
         console.log("drawImage returned", ctx);
     }).catch((e) => {
         console.log(e);
     });

Then I got the following error:

Looking forward to your help, thanks!

narmadaravali · July 20, 2022, 6:30pm

Can you try this approach, wait for inCamera app to load first, then postMessage to the sidebar. Then drawImage, you might also have to drawWebView to see the result.

wei.hu · July 21, 2022, 3:53pm

Hi~ Narmada, I have try this approach, but still get an error as below

If you can give me an example, I will be very grateful!

narmadaravali · July 22, 2022, 7:48pm

Hi, this example should work

function getMyImageData(width, height) {
    const canvas = document.createElement("canvas");
    canvas.width = width;
    canvas.height = height;

    const img = new Image(100, 200);
    img.src = zoomlogo; // insert any image here

    const ctx = canvas.getContext("2d");
    ctx.drawImage(img, 0, 0, width, height);
    return ctx.getImageData(0, 0, width, height);
  }

async function testDrawImage() {
    const imageHeight = 100;
    const imageWidth = 500;
    const imageData = getMyImageData(imageWidth, imageHeight);
    await zoomSdk
      .drawImage({
        imageData: imageData,
        x: 0,
        y: 720 - imageHeight,
        zIndex: 2,
      })
      .then((result) => {
        console.log(result);
      })
      .catch((error) => console.error("Unable to call drawImage ", error));
  }

output:

wei.hu · July 24, 2022, 1:29pm

Hi, Narmada Ravali, thanks for you help, I try it with your example, but got the same error again, Is it because I missed some other configuration?

below is the full code:

import React from "react";
import Button from "@mui/material/Button";
import zoomSdk from "@zoom/appssdk";
import { IMAGES } from "./constants";

const MY_IMAGE = "https://9bae041b1da7.jp.ngrok.io/assets/1@2x.jpg";

const CameraMode = () => {
    const Start = async () => {
        const configRes = await zoomSdk.config({
            size: { width: 480, height: 360 },
            capabilities: [
                // The following are needed for Layers API.
                // Include any other capabilities your app needs here, too.
                `getRunningContext`,
                `runRenderingContext`,
                `closeRenderingContext`,
                `drawParticipant`,
                `clearParticipant`,
                `drawImage`,
                `clearImage`,
                `drawWebView`,
                `clearWebView`,
                `postMessage`,
                `sendAppInvitationToAllParticipants`,
                `onMessage`,
                `onMyMediaChange`
            ]
        });

        console.log("configRes", configRes);

        const renderingCtx = await zoomSdk.runRenderingContext({ view: "camera" });

        console.log("runRenderingContext returned", renderingCtx);

        function getMyImageData(width, height) {
            const canvas = document.createElement("canvas");
            canvas.width = width;
            canvas.height = height;

            const img = new Image(100, 200);
            img.src = MY_IMAGE; // insert any image here

            const ctx = canvas.getContext("2d");

            ctx.drawImage(img, 0, 0, width, height);
            return ctx.getImageData(0, 0, width, height);
        }

        async function testDrawImage() {
            const imageHeight = 100;
            const imageWidth = 500;
            const imageData = getMyImageData(imageWidth, imageHeight);
            await zoomSdk
                .drawImage({
                    imageData: imageData,
                    x: 0,
                    y: 720 - imageHeight,
                    zIndex: 2
                })
                .then((result) => {
                    console.log(result);
                })
                .catch((error) => console.error("Unable to call drawImage ", error));
        }

        testDrawImage();
    };

    return (
        <div>
            <Button
                onClick={() => {
                    Start();
                }}
            >
                Start
            </Button>
            <canvas id="canvas" />
            <img src={MY_IMAGE} style={{ display: "none" }} />
        </div>
    );
};

export default CameraMode;

coreyhobbs · July 26, 2022, 5:42pm

Are you using a free NGROK account? Also, are you on MAC?

wei.hu · July 27, 2022, 1:45am

I use a paid NGROK, on my MAC, the Immersive mode is work for me, and other API can also work, except drawImage in Camera Mode

wei.hu · July 29, 2022, 9:05am

It is work for me now, thanks everyone

narmadaravali · July 29, 2022, 5:41pm

Hi Wei, what changes did you do? Glad you are unblocked.

wei.hu · July 31, 2022, 3:08am

Hi Narmada Ravali, I call zoomSdk.config and zoomSdk.runRenderingContext methods in useEffect hook, then draw a image when user select a image. and it can draw success, some code as below:

// App.tsx
useEffect(() => {
    const initial = async () => {
        if (!isZoomApp) {
            return;
        }

        const configRes = await zoomSdk.config({
            size: { width: 480, height: 360 },
            capabilities: [
                // The following are needed for Layers API.
                // Include any other capabilities your app needs here, too.
                `getRunningContext`,
                `runRenderingContext`,
                `closeRenderingContext`,
                `drawParticipant`,
                `clearParticipant`,
                `drawImage`,
                `clearImage`,
                `drawWebView`,
                `clearWebView`,
                `postMessage`,
                `sendAppInvitationToAllParticipants`,
                `onMessage`,
                `onMyMediaChange`,
                `onReaction`
            ]
        });

        await zoomSdk.runRenderingContext({ view: "camera" });
    };

    initial();
}, []);

useEffect(() => {
    let src = currImg[size];
    if (showOverlays && src) {
        drawImg(src).then(({ imageId }) => {
            drawImgRef.current = imageId;
        });
    } else {
        zoomSdk.clearImage({
            imageId: drawImgRef.current
        });
    }
}, [showOverlays, currImg, size]);

// utils/index.ts
const CANVAS_WIDTH = 1280;
const CANVAS_HEIGHT = 720;

const drawImg = (
    src: string
): Promise<{
    imageId: string;
}> => {
    const canvas = document.createElement("canvas");
    const ctx: CanvasRenderingContext2D = canvas.getContext("2d") as CanvasRenderingContext2D;
    canvas.width = CANVAS_WIDTH;
    canvas.height = CANVAS_HEIGHT;

    // coordinate reference adjustment
    let scale = -1;
    ctx.translate(canvas.width, 0);
    ctx.scale(scale, 1);

    const img = new Image();
    img.src = src;
    ctx.drawImage(img, 0, 0, img.width, img.height, canvas.width - img.width, 0, img.width, img.height);

    // coordinate reference restoration
    ctx.setTransform(1, 0, 0, 1, 0, 0);

    const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);

    return zoomSdk.drawImage({
        imageData,
        x: 0,
        y: 0,
        zIndex: 2
    });
};

soham.gada20 · February 1, 2024, 10:17am

I am facing the same issues can you please share the file where you made these changes or if possible can you please send the github link for this application

petercox1002 · February 16, 2024, 6:02pm

It seems like there might be an issue with the image source or the rendering context in camera mode. Double-check the image URL, and ensure that the rendering context is properly initialized.

Topic		Replies	Views
drawImage not working immediately after runRenderingContext Zoom Apps	1	387	October 15, 2022
drawImage not working in ImmersiveMode and zoom apps keeps crashing Zoom Apps	0	277	November 12, 2023
The zoom client encountered an error while processing the request! Zoom Apps	11	733	August 17, 2022
drawParticipant in Camera Mode returning an error Zoom Apps	3	517	June 27, 2022
Cannot get Camera Mode to work in Layers API Zoom Apps	25	1304	October 6, 2023

drawImage in Camera Mode returning an error

Related Topics