Where do I find private messages? I was keeping an eye on my Zoom Developer Forum notifications and my email but I never saw anything about a private message.
We also checked and our certificate chain is fine.
This is hardly a certificate issue on our sides.
If that were the case, all webhook events would be missing.
Also, it wouldn’t explain why it starts working again after an OAuth reconnect.
Could some of your webhook-sending worker nodes be having an issue with a certificate?
This could potentially explain the non-consistent behavior with this (as maybe reconnecting via OAuth assigns another webhook node with a working certificate - although it would be strange, why it would stop working in a day or a week).
Another related thread:
thank you. Would you be able to inform us of any progress made so far?
Also, is there a way for us to access the webhook logs in any way? API is fine.
I will let you know as soon as I hear back from Engineering.
Here is the Get webhook logs endpoint that could help you monitor this issue
Update: we heard from someone in Zoom engineering that this issue isn’t happening to other customers (at least not broadly), so that it is likely on our end.
I don’t know exactly how it is happening on our end, but I decided to try putting Cloudflare in front of our site so that Zoom webhook requests would be dealing with SSL certs on their edge nodes instead of certs in our system.
So far this seems to be working. It’s too early to tell for sure that the issue is fixed, but we haven’t received any errors since switching.
@cedric-swivvel
Interesting. Is it still working fine for you with the CloudFlare workaround after a day or two?
It’s interesting that it would work initially, then stop working in a day or after a couple of days.
And again start working after an OAuth reconnect.
What could cause such inconsistent behavior @elisa.zoom ?
Seems to still be working. We don’t see any new errors in the /webhook_logs endpoint since putting Cloudflare in place.
I still don’t understand what was up with our certs, though. They were Let’s Encrypt certs, so they should have been accepted with no problem. And I have no clue why some requests would get a cert error and some wouldn’t, since every request should have been served the same cert. And I don’t think this could have been a problem with specific servers within our VPC since there was no indication in our logs that the Zoom webhook requests were getting past the SSL handshake, which should have a single point of entry, so it’s not like some subset of servers would be serving an invalid cert. And we weren’t getting these errors with end user requests or any webhook requests from multiple other third parties.
But since it’s fixed now we’re going to stop digging, so hopefully the solution of putting some proxy in front of the site can work for anyone else encountering this problem, if it’s an option.
Thank you @cedric-swivvel
We did prepare everything, but need to change the URL of the Webhook, since we can’t transfer our root domain to Cloudflare. This requires the app to be submitted again, and last time this happened, all out customer’s webhooks got invalidated, so we’re a bit scared to go through that again.
We’re also using Let’s Encrypt certificates on Heroku.
There were reports with their certificates before as well:
@elisa.zoom can you please confirm, that updating the Webhook URL (to the one on CloudFlare) and submitting the app will not disconnect all connected account’s webhooks?
Hey @jenniferb
Updating the endpoint url won’t cause your current users any disruption and they wont need to reauthorize the app again
Hi @elisa.zoom,
this is not what I’m asking. There was a serious bug earlier this year on Zoom side, when we last had to submit our app for your TDD review, and it broke all our customer’s webhooks, so they all had to reconnect.
I’m asking if this was fixed so that we can try resubmitting our app (without upsetting all our users) with a new URL that is proxied through CouldFlare?
Hello, also experiencing a similar issue- We don’t receive any events after first connecting with oauth, but for some users it works fine after disconnecting then reconnecting. When testing with newly created zoom accounts we do not receive events even after reconnecting.
1 Like
Hi @jenniferb
I am adding my friend and colleague @kwaku.nyante to this thread, he is in the Marketplace team and can guide you with the update request
Our App’s Webhook URL change was now approved (thank you @elisa.zoom and @kwaku.nyante for fastracking this!) and we’re now proxying through CloudFlare.
We will observe.
@elisa.zoom going through CloudFlare can not be the accepted solution, unfortunately (I don’t know who marked it as the solution), this is just a test. And you can see there are several other people reporting the same issue. If it was a certificate issue, why would be initially work, then suddenly stop working? And only for some users?
We would still like to hear from your Engineering team why this is happening. There are certainly no timeouts on our side, because the request does not reach our servers (all requests are logged on load balancer level).
Hi @jenniferb
I understand that, as I mentioned in the private thread, for the specific case that you shared with me and that they investigated, it was a timeout issue. If you have any other example, feel free to share that in the private thread and I will create a ticket for you