The speaker sometimes does not work when using Puppeteer with a fake media device

Video SDK Web
22

Hi @lmtruong1512

Should we call startAudio immediately after joining the session, or wait for a specific event?

It depends on your use case, but it’s recommended to listen to the auto-play-audio-failed event to handle failures caused by browser privacy policy.

Are there any potential issues, and what would be the recommended approach for this scenario?

These two methods return promises, so you need to wait for the promises to resolve before calling the next method. It’s recommended to add await.

Thanks
Vic

1 Like
23

@vic.yang Thanks for your help. Could you take a look at this as well? Below is a summary of the issue we’re facing:

  1. During some meetings, certain users (referred to as “BOTs,” which run on headless browsers using Puppeteer) are unable to hear the other speaker. The affected session IDs are:
9aF8YPhEQgaHxr4hoZwG3w==
UsIoxcsDSgerPxuK2NZwIQ==
  1. Upon analyzing the logs, you found that these two BOT users did not successfully start audio, possibly due to the browser’s autoplay policy requiring user interaction.

Questions:

  1. If the issue is related to the browser’s autoplay policy, we’re curious why it occurs randomly and not consistently across all BOT ? (Each BOT runs on an EC2 instance with the same configuration, logic code, and separate sessions)

  2. We’ve only been able to reproduce the problem when forcing the override of HTMLMediaElement.prototype.play to always throw error. However, in practical scenarios, we’re unable to reproduce it again. Do you have any suggestions on how we might reproduce this issue? (We need to reproduce the issue to ensure we’re addressing the correct root cause and to validate that the fix is effective)

Thank you in advance!

24

Hi @lmtruong1512

9aF8YPhEQgaHxr4hoZwG3w==
UsIoxcsDSgerPxuK2NZwIQ==

In order to further analyze this issue, could you report the detailed log to Zoom? You can use the following method to report it:

const clientSideTelemetry = client.getLoggerClient()
clientSideTelemetry.reportToGlobalTracing()

Thanks
Vic

1 Like
25

@vic.yang Sorry, we haven’t been able to reproduce the issue to capture detailed logs again. Do you have any suggestions for how we might reproduce it again?

26

Hey @lmtruong1512

Sorry, we haven’t tested the BOT case, but in the browser, opening a new incognito window and then programmatically calling startAudio immediately after joining the session can reproduce this issue. The root cause is the auto-play policy.

Thanks
Vic

27

@vic.yang Thank you for your support. We followed these steps but were still unable to reproduce the error:

  1. User A creates and joins a meeting with the microphone on.
  2. In an incognito window, user B joins the meeting and starts audio immediately.

In this scenario, User B is still able to hear User A without any user interaction.

Could you confirm if the steps above are correct?
Also, thank you for confirming that the root cause is related to the autoplay policy — we’ll focus our investigation in that direction.

28

Hi @vic.yang,
Could you please check the session VyRxA26AQsSr/A4nch70IA==? We’re currently experiencing the same error again — no speaker or microphone, and the BOT user is unable to hear others speaking.

  1. Could this be related to the browser’s autoplay policy requiring user interaction again?
  2. Can you check the logs to confirm that startAudio() was retried 3 times, each time after a 40-second, that all attempts resulted in a timeout error? The speaker sometimes does not work when using Puppeteer with a fake media device - #16 by vic.yang

Thank you very much!

image
image1516×116 15.7 KB

29

Hi @lmtruong1512

VyRxA26AQsSr/A4nch70IA==

UserID: 16782336 didn’t join audio and UserID: 16784384 joined audio with speakerOnly.

Since we don’t have detailed logs, I can only suspect that UserID: 16782336 failed to join audio due to the browser’s autoplay policy.

Thanks
Vic

1 Like
30

Hi @vic.yang, sorry to bother you again. Could you please help us check the latest BOT user that joined this session: BtO+9ZrWS+uneAТ+mipkqw==

  1. The speaker appears to be blank again. Can you check whether startAudio was triggered? If it were, may it fail due to the browser’s autoplay policy??

  2. It seems the BOT is not listening to the connection-changed or user-removed events when another user leaves the meeting or when the host ends the session for everyone. Could you check whether the BOT on the user’s client is successfully listening for those events?

  3. Is it possible that the BOT is frozen on the client side, which is why it’s not performing actions or responding to events as expected?

We’re in the process of adding telemetry to report detailed logs to Zoom, so we’ll have more information to share with you in the future. However, for now, could you please help confirm the points mentioned above? Thank you very much!

31

Hey @lmtruong1512

BtO+9ZrWS+uneAТ+mipkqw==

We searched the session ID in our system but couldn’t find any session that matches it.

Thanks
Vic

32

@vic.yang
Sorry, the Unicode characters might be incorrect. Could you please help me check this session ID “BtO+9ZrWS+uneAT+mipkqw==” ?

33

Hi @lmtruong1512

BtO+9ZrWS+uneAT+mipkqw==

Could you help check which user ID corresponds to the BOT user—16778240 or 16782336?

P.S. We’re unable to view the user name since it is considered PII data.

Thanks
Vic

34

Thank you, @vic.yang. Could you please check the user associated with ID: 16782336?

35

Hi @lmtruong1512

BtO+9ZrWS+uneAT+mipkqw==

UserID: 16782336

The user didn’t start the audio.

Thanks
Vic

1 Like
37

@freelancer.nak Thank you for your supportive response. We’ll consider this fix if the current one doesn’t work.

38

Hi @vic.yang,

We’ve added telemetry to report detailed logs to Zoom. Could you help check what happened with the BOT user that caused the blank speaker issue on the management page?

Here are our session IDs that have blank speaker:

ivmiYDxrTIqXRWTSwzzoTg==
GrO2QNNJSEO7EYP/BercYQ==
KB3kkx/iQNy31HpTU86DFQ==

In these cases, our logs did not show the BOT receiving the auto-play-audio-failed event, nor did we find any startAudio errors.
Could there be another underlying cause unrelated to the browser’s autoplay policy?

Thanks!

39

Hi @lmtruong1512

ivmiYDxrTIqXRWTSwzzoTg==
GrO2QNNJSEO7EYP/BercYQ==
KB3kkx/iQNy31HpTU86DFQ==

UserID: 16781312

From the detailed logs, it appears that this user never called the startAudio method. Could you please double-check your call logic on your side?

Thanks
Vic

1 Like
40

@vic.yang
Thank you for your feedback.
Could you please check the logs for these session IDs to see if the BOT users called the startAudio function?
I noticed that the Zoom dashboard shows a blank speaker for those sessions.

gg91sLAPROWmlBCzVkGECw==
spjLh/GMSlCNXlTOpGGBNw==
20UNVw9lTOS588xZ+wbNmg==
cg3S3UQBS0Svh26WOMg0Jw==

Thanks
Phuong

41

Hi @phuongnv1

gg91sLAPROWmlBCzVkGECw==
UserID: 16781312

spjLh/GMSlCNXlTOpGGBNw==
UserID: 16782336

20UNVw9lTOS588xZ+wbNmg==
UserID: 16782336

cg3S3UQBS0Svh26WOMg0Jw==
UserID: 16781312

The above users didn’t call the startAudio method.

spjLh/GMSlCNXlTOpGGBNw==
UserID: 16784384

The user called the startAudio method and was able to hear others.

Thanks
Vic

2 Likes
42

Thanks for sharing the details. I agree, the random behavior is puzzling; let’s dig into consistent reproduction steps so we can confirm if autoplay policy is truly the cause.