Events received with excessive delay/dropped from OAuth app with Websocket subscriptions

I’ve been using an OAuth server to server app with two subscriptions over Websocket to test an integration with Zoom Contact Center. Up until a few days ago I stopped receiving events for the user I’m testing. I do however get from time to time events for other users with the same account id and IF I receive an event for the user under test, it arrives hours later.

The app is configured as follows:
------ Event Subscription #1

  • Method: Websocket
  • Events (46) [All User activity selected, Several Contact Center events selected (Types: Task, User, Engagement, Recording)]
    ------ Event Subscription #2
  • Method: Websocket
  • Events (46) [All User activity selected, Several Contact Center events selected (Types: Task, User, Engagement, Recording)]
    ------ Scopes: Selected all from User, Account and Contact Center

I can establish the WSS connection via postman and it’s kept alive by periodically sending the {“module”: “heartbeat”} to which I receive a response {“module”:“heartbeat”,“success”:true,“content”:“live”,“header”:{}} which means that I can receive events over the websocket.

Troubleshooting steps taken so far:

  • I’ve updated the Zoom desktop app for Windows to the latest version (5.15.12 21574)
  • I’ve quit the app, signed out/in.
  • Deactivated/reactivated oauth app.
  • I’ve assigned more scopes to the app.
  • I’ve also queried the specific user via the /users API to make sure that the account _id matches which it does.

So far today I’ve only seen 1 event for my user even when I’ve made several calls, changed states, logged it in/out, etc. Is there something I can do to figure out what’s going on?

I’ve just received another example of a delayed event. It arrived on the Websocket at 14:58:19.429 EST, but the event is from ~2h ago (“event_ts”: 1694539182464 which is 13:19:42.464 EST). JSON received below.

{
“module”: “message”,
“content”: {
“event”: “contact_center.user_logout”,
“payload”: {
“account_id”: “ACC_ID”,
“object”: {
“user_id”: “USER_ID”,
“display_name”: “Alejandro”,
“user_email”: “USER_EMAIL”,
“date_time”: “2023-09-12T17:19:42Z”
}
},
“event_ts”: 1694539182464
},
“header”: {
“Authorization”: “AUTH_ID”,
“clientId”: “CLIENT_ID”,
“x-zm-trackingid”: “TRACKING_ID”,
“x-zm-signature”: “SIG”,
“x-zm-request-timestamp”: “1694545099”
}
}

Hi @alejandro.flores ,

This is unexpected behavior. I would like to investigate the root cause of this.

Can you please email me these details so that we can investigate?
I am dming you my email.
Thanks

Hi @ojus.zoom,

I have a websocket app that seems to display a similar issue. At the moment, I am seeing approximately a 2 hour delay in the events received.

Anything that can be done on our end?

Thanks

It seems you’re facing a challenging issue with your OAuth server-to-server app’s integration with Zoom Contact Center over Websockets. When troubleshooting such complex scenarios, it’s essential to explore different angles to pinpoint the problem. Here’s a step-by-step approach to help you figure out what’s going on:

  1. Check Zoom Status Updates: Begin by checking the official Zoom status page. Sometimes, issues may be on Zoom’s end, affecting the event delivery. Ensure that their systems are running smoothly.
  2. Inspect Error Logs: Examine the logs on both your OAuth server and Zoom’s side. Look for any error messages or exceptions related to event subscription or Websocket connections. This can provide valuable insights into what’s failing.
  3. Rate Limiting and Event Volume: Ensure you’re not hitting any rate limits imposed by Zoom’s API. It’s possible that if you exceed certain limits, events might be delayed or not delivered at all. Review your event volume and make sure it’s within acceptable limits.
  4. Verify Event Subscription Setup: Double-check your event subscription settings. Ensure that the selected events, scopes, and subscriptions are correctly configured in your Zoom app dashboard. Any mismatch here can lead to missing events.
  5. Review WebSocket Connection Handling: Assess how you’re handling WebSocket connections. Confirm that your WebSocket connection is stable and doesn’t experience frequent disconnects. You might want to log connection status changes to track any anomalies.
  6. Inspect User-Specific Issues: Since you’re experiencing this issue for a specific user, focus on their account. Check if there are any unusual settings, permissions, or restrictions associated with this user’s account that might affect event delivery.
  7. Debugging with Websocket Frames: If possible, use tools or libraries that allow you to inspect WebSocket frames exchanged between your app and Zoom’s servers. This can help identify any anomalies or issues with the communication itself.
  8. Engage Zoom Support: If the issue persists and you can’t identify the root cause, consider reaching out to Zoom’s developer support. Provide them with all the details you’ve gathered during your troubleshooting. They might have insights or solutions specific to their platform.
  9. Monitoring Over Time: Continue monitoring the situation over a period to see if the delayed event issue persists or resolves itself. Sometimes, these issues can be transient.

Remember that troubleshooting complex integrations can be a gradual process. By systematically exploring these steps and gathering as much data as possible, you’ll be better equipped to identify and resolve the issue affecting your events from Zoom Contact Center. Good luck, and I hope you find a solution soon! :hammer_and_wrench::mag::globe_with_meridians:

@kyl3 @alejandro.flores @cstevensleisa

It has been over two weeks since I last attempted to run a WebSockets application and im failing to receive any events. Code worked fine to receive events prior to that time. Currently my code receives the build_connection event and all heartbeat acknowledgements, but not a single phone event. The Server to server OAuth app in question has every single available scope for testing purposes and every single phone event subcribed to with zero call events coming through.

Same results with PostMan as well.

This “beta” is more like a fantasy for me than actual functioning API.

We keep asking for an ETA, but so far no one’s offered a timeline for the fix.

1 Like

Hi Team,

We apologize for the inconvenience. We currently do not have an ETA on this fix, but we will communicate as soon as we have an update.

I highly doubt you will communicate anything. I replied to you 8days ago about the GraphQL roadmap and you ignored me.

@ojus.zoom, just checking in on this. Any updates by chance?

1 Like

Also asking for an update if maybe more activity will actually get you to look at this?

@ojus.zoom

We met with Ojus a few weeks back and we were told that sometime around mid November they were going to allocate more servers to handle the Websocket load, so we’re waiting for that. Another long term fix would come sometime next year (February timeframe).

We’re waiting for the upcoming change to see if it’s stable enough for us to continue dev/testing.

1 Like

Thanks for sharing that update with us, @alejandro.flores . Much appreciated!

1 Like

Apparently Zoom should hire you to provide better communication with us… Thanks a lot. Glad to hear they are in fact working on something. I did a test last night and I did seem to get a few phone call events, so i’m hopeful again.

Sorry for the delay here. We are rolling out a fix in November, which should be able to handle more traffic for websocket events

1 Like

@ojus.zoom, any updates on if the fix was ever published?

@kyl3 Yes, we did increase the number of servers that were handling the websocket traffic. Are you still seeing significant delays?

@ojus.zoom , sorry for the delayed response. Was able to do some testing today and things look much more responsive. I’ll keep an eye out for issues as I work on my project and let you know if anything else comes up.