Zoom Cloud Recording Transcription Behavior with Hindi/Code-Switched Audio

API and Webhooks Meetings
,
1

Query:

I’m conducting a quality analysis of Zoom Cloud’s automatic transcription service for code-switched audio (English-Hindi mixing, commonly called “Hinglish”).

Observed Behavior: During transcription of a meeting with mixed English-Hindi speech, I noticed:

  1. Hindi words are frequently deleted - Words like “matlab”, “thoda sa”, “tha”, “ka” are completely missing from the transcript
  2. Attempted phonetic transliteration - Some Hindi words are converted to English phonetics:
  • “ki” → “key”
  • “aapne jo” → “aapneemi bowlanger”
  • “sad” → “sadka”
  • “hota” → “hotel”
  1. Entire Hindi phrases dropped - Multi-word Hindi expressions like “aapne jo hindi me bola hai jo ground truth tha” are completely omitted

Questions:

  1. What is Zoom Cloud’s expected behavior when encountering non-English (specifically Hindi) words in an English-language meeting recording?
  • Does it attempt phonetic transliteration?
  • Does it skip/ignore Hindi words?
  • Can I enable a specific language setting for code-switched content?
  1. Is there a configuration option to handle code-switched/multilingual audio better?
  • Can I set primary + secondary language detection?
  • Is there a Hinglish or code-switching mode?
  1. Language detection: Does Zoom Cloud automatically detect and switch between languages, or does it stick to the meeting’s primary language setting?

Goal: Understand if this is expected behavior or if there’s a configuration to better handle code-switched audio.

2

hi @Abu ,

That is not a question I think we can reply to as developers. We may be able to hunt down a Product Manager.

What I can say from personal experience is that these translations are never going to be as good as a professional translator who understands words and context in real time. For mission critical situations, you should always find a human translator. Zoom is well setup for allowing translators to have their own audio channel for attendees to listen to.
There are also third party companies who can provide more powerful and extensive translation services.
Zoom is not good at then recognising changes in language, and not really expected to. THe options are there to show the language you want to speak or translate to. This is most noticable when non-English languages do not have equivilents and use English words or phrases in every day use.
There are options to upgrade the translation languages.

To your Goal: there is no real system for code based switched audio. There is no code option to change this during a meeting. Indeed Captions do not even declare the language they are presenting.

@elisa.zoom is there anything else to add?

John