How can I get live transcription text using Meeting SDK (Windows)

Description

I cannot get the Live Transcription text. Could you please advise on how to obtain it?.

The IClosedCaptionControllerEvent::onLiveTranscriptionMsgInfoReceived() callback method is not being called.

Which version?

Meeting SDK for Windows (C++)

zoom-sdk-windows-5.15.5.19455

To Reproduce (If applicable)

  1. Download and build the “sdk_demo_v2.sln” on your Windows machine. Note: select “Release/Win32” build mode.
  2. Have another ZOOM client (host) ready to start the meeting on another machine.
  3. Start a Zoom meeting.
  4. Run sdk_demo_v2 to join the meeting.
  5. In the first dialog, check “Customized UI Mode”. Select the “SetDomain” button.
  6. Create a JWT token, then copy and paste the token into the appropriate field and click the “Auth” button.
  7. In the second dialog, select the “Only Join” tab window and type “Meeting number”, “Name” as Bot’s name, and “Meeting password”. Then, select the “join” button.
  8. You should now see your meeting bot in the Zoom client on the host machine.
  9. Activate “Live Transcription” on the host Zoom client.
  10. Say something and ensure that the live transcriptions appear as subtitles.
  11. Set a breakpoint in the SDKInterfaceWrap::onLiveTranscriptionMsgInfoReceived() callback method.
  12. This callback method is not being called.

Device (please complete the following information):

  • Device Spec: DELL Note PC
  • OS: Windows 10 Home
  • IDE: Microsoft Visual Studio Community 2019 Version 16.11.27
    • Microsoft .NET Framework Version 4.8.09037
  • zoom client version on another machine: 5.15.7 (21404)

Additional context

I have implemented onLiveTranscriptionStatus() in the same context, and then it is working successfully when subtitles are disabled or enabled by the host. Based on this, I believe the Meeting SDK has been implemented correctly. However, onLiveTranscriptionMsgInfoReceived() is not called. I would like to know how this can be called and how I can get the live transcription.

Note: I have not checked the onLiveTranscriptionMsgReceived() callback, as it is currently treated as deprecated.

https://marketplacefront.zoom.us/sdk/meeting/windows/deprecated.html

Hi @daijufujita ,

I’ve just tested this, and here are some high level steps which I’ve gotten this to work. I’ll share more refined details if you need them.

For onLiveTranscriptionMsgInfoReceived callback to activate, you will need to enable some of these

	        captionController->StartLiveTranscription();
			captionController->SetMeetingSpokenLanguage(0);
			captionController->SetTranslationLanguage(1);
			captionController->EnableReceiveSpokenLanguageContent(true);

I’ve called these as a host for now.

1 Like

Hello @chunsiong.zoom ,

Thank you very much for your response.

I’ve called the following methods as a host:

captionController->StartLiveTranscription()

  • This is working successfully.

captionController->SetMeetingSpokenLanguage(0)

  • This is also working successfully.

However, I’m experiencing issues with the following two methods:

captionController->SetTranslationLanguage(1)

  • This resulted in an error: SDKERR_MEETING_DONT_SUPPORT_FEATURE.

captionController->EnableReceiveSpokenLanguageContent(true)

  • This returned an error: SDKERR_INTERNAL_ERROR.

The OnLiveTranscriptionMsgInfoReceived() callback is not being called.

Could you share more detailed information to help resolve these issues?

Thank you in advance for your assistance.

@daijufujita

i’m still testing this out.

It seems that the error which you have encountered are related to the account features.
The account which you have tested on might not have the live transcription and translation feature enabled.

Let me get back to you if there are alternatives methods which might trigger OnLiveTranscriptionMsgInfoReceived

Meanwhile could you try to observe whats the behavior of this if you set the value to -1?
captionController->SetTranslationLanguage(-1)

Hi @chunsiong.zoom, just wanted to say thank you and that the demos found in your github have been immensely helpful in getting us started for our projects. We’re also trying to get live transcription messages and have been running into the same issues with onLiveTranscriptionMsgInfoReceived not being triggered. In the meantime, we’ve tested onLiveTranscriptionMsgReceived and that seems to be working fine still. Would you recommend just sticking with that for the time being even though it’s being deprecated soon?

Update: After some further testing, we’re still unable to trigger onLiveTranscriptionMsgInfoReceived, but we did notice that onOriginalLanguageMsgReceived is being triggered as long as live transcription is turned on (captionController->StartLiveTranscription()). This event also supplies a ILiveTranscriptionMessageInfo object so it seems to be working pretty much in the same manner as onLiveTranscriptionMsgInfoReceived. In the sample code, though @chunsiong.zoom, you have marked onOriginalLanguageMsgReceived as deprecated, but the latest SDK docs hasn’t. Is this method safe to use until we figure out how to properly trigger onLiveTranscriptionMsgInfoReceived?

1 Like

@xschen ,

that was wrongly marked.
the deprecated method is onLiveTranscriptionMsgReceived

I’m currently doing some testing, and will share more information in a bit

@xschen

Thank you for replying so quickly and for testing this issue.

captionController->SetTranslationLanguage(-1)
returns
SDKERR_MEETING_DONT_SUPPORT_FEATURE(20)

Thank you.

@daijufujita @xschen ,

I’ve concluded some testing and here are some additional notes which you might find useful.

  • onOriginalLanguageMsgReceived is the replacement method for onLiveTranscriptionMsgReceived
  • If you do not have a translation plan in place, can call the following code, to get the speech to text in onOriginalLanguageMsgReceived
    captionController->StartLiveTranscription();
    captionController->SetMeetingSpokenLanguage(0); //english
    captionController->SetTranslationLanguage(-1); //none
    captionController->EnableReceiveSpokenLanguageContent(true);
  • If you have a translation plan in place, below is the variation in code, to get the translated speech-to-text in OnLiveTranscriptionMsgInfoReceived
    captionController->StartLiveTranscription();
    captionController->SetMeetingSpokenLanguage(0); //english
    captionController->SetTranslationLanguage(1); //mandarin
    captionController->EnableReceiveSpokenLanguageContent(true);
1 Like

Thanks @chunsiong.zoom for the additional notes. We do have the translation plan for our accounts, but I do remember using all those settings earlier today and OnLiveTranscriptionMsgInfoReceived still wasn’t triggering. Maybe it was because I wasn’t doing it as host, but I can give that a try tomorrow. Regardless, onOriginalLanguageMsgReceived is working and we don’t really need any translation, so we may just stick with that. On a slightly separate note, it’s a not a big issue and not sure if it’s intended, but we’re also noticing that each transcription or caption message is triggering their respective events twice. Even after I conditioned it to only trigger on ILiveTranscriptionMessageInfo objects of type ZoomVideoSDKLiveTranscription_OperationType_Complete, the events are still being triggered twice.

@chunsiong.zoom
@xschen
Thank you very much!
I have obtained the transcription data.

1 Like

@chunsiong.zoom
Hello Siong,

I can get transcriptions with the bot as the host now, but I would like to get transcriptions when the bot is not the host. Please support me on how can I get.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.