We’re looking to build a conversational voice assistant that can join Zoom meetings and take notes actions etc. It will need to listen and it will also need to speak.
We’re struggling to validate how we can have our AI assistant speak in a zoom meeting we have added it to.
Can anyone share any points?
Hey @thewelshbeuller,
Thank you for reaching out to the Zoom Developer Forum. This sounds like a good use case for our upcoming Zoom feature called Zoom Apps. You can sign up at that link to make sure you’re kept in the loop.
For now, it seems the best method to accomplish something like this is to have your AI join the meeting as a participant. It could then process the meeting audio and listen for voice commands. You could also have the AI use a virtual microphone to allow the AI to speak in the meeting.
The biggest blocker is that with the Web SDK, there isn’t a method to automatically join audio or video. Due to security and privacy restrictions implemented by the browser and Zoom, user interaction is required to enable microphone or camera devices even if they are virtually created by your AI.
You may be able to use a Native Client SDK (macOS or Windows) to join the meeting audio/video programmatically but I wasn’t able to confirm this in our documentation. Using a native SDK will also provide you with better access to compute resources available to the device.
When it comes to questions about our Windows or MacOS SDKs, I recommend posting in our #desktop-client-sdk:windows or #desktop-client-sdk:macos categories. They’ll be able to advise further on that topic.
I hope that helps! Let me know if you have any questions.
Thanks,
Max
Thanks for your note. So checking, are you saying this can be done going forward with zoom apps?
On the note of getting just the audio, how do all of the current transcription services capture the audio?
Hey @Beuller,
I’m not positive if Zoom Apps will provide access to the audio data, I’m still working to confirm this. However, if you want to have an assistant join the meeting, using a Zoom App will definitely be the best method.
Custom Closed Captioning services make use of the closed captioning API that we have provided. You can see more information on integrating with a custom service here.
I hope that helps!
Thanks,
Max
Morning! Any idea when Zoom Apps is coming for release?
Hi @thewelshbeuller,
We expect Zoom Apps to be released within the next month or two. Please stay tuned to developers.zoom.us for updates.
Thanks,
Will
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.