Randomly receiving "Invalid access token" for Server-to-Server OAuth

Starting this week, we have started receiving a lot of errors when calling Zoom APIs with our Server-to-Server OAuth token.

{ "code": 124, "message": Invalid access token." }

When I look at the access token in question, it has not expired…there’s plenty of time left. We also have logic in place to refresh the token well before it’s scheduled to expire.

Has something changed about the way Server-to-Server OAuth access tokens are managed on the Zoom side? We’ve had to put in a retry mechanism for all our calls to zoom (regenerating the token in between) and it’s been tripping quite frequently this week, whereas before it almost never was invoked.

Thanks!
David

1 Like

Hi @david5
I have not heard of this issue before and I have not experienced this behavior.
Feel free to send me the email associated with your account via Direct Message so I can look into this issue.

Best,
elisa

If there are multiple concurrent processes that are making API calls, they might be invalidating a token that another process is currently using. You can apply for access to a workaround in Server-to-Server OAuth: annoying token-invalidation.

1 Like

Thanks for chiming in @MultiplayerSession

Are you saying that if I have multiple instances of my service running (as one would expect for a scalable, redundant application), that I can only have one valid access_token across all those instances? In other words, each time I call https://zoom.us/oauth/token?grant_type=account_credentials&account_id=[account id], it invalidates the previous access token that was issued?

If this is the case, how are we supposed to build a clustered application that makes API requests in the Server-to-Server OAuth case? I don’t really want to have to keep the access token in a single place and try to synchronize it across all the instances of my application.

Hi @david5
Yes, for the moment that is the behavior we have.
When you request a new access token, the previous one will be invalidated. We do have a workaround for now and is that we can increase the index of your app (it is 0 by default).

So for example, if the index of your app is increased, an access token generated at index 0 will work independently of a token generated at index 1 or 2 and you can specify the index for the token by appending the “index” query parameter to your token request.

If you are interested in this, please go ahead and create a ticket with our Developer Support team here:

and share with me the ticket ID and I will take it from there!
Thanks
Elisa

Thanks @elisa.zoom . I’m not sure I understand how increasing the index of my app helps matters any. My server-side application is a cluster of 8 instances running. Each of those instances use our secret keys to request an access token and maintains it in memory before requesting a new one (before the 1-hour expiration). It sounds like with this new proposal, each instance would have to be configured with a unique index number, which it could pass to the zoom authentication endpoint. In order to do this, I’d have to use some kind of shared storage to make sure that each service didn’t pick a number already in use. If I wanted to scale up, there’s the potential that I’ll run out of index numbers.

This all feels very strange for an authentication mechanism that was intended for the server side. What am I missing here? Is this not the way that Server-to-Server OAuth is intended to be used?

I submitted a request to add a block of index values, as requested.

https://support.zoom.us/hc/en-us/requests/15140779

Hi @David

Thank you for sharing more details with me.
So yes if each of your instances requests an access token with the same credentials (same Server-to-Server OAuth app) then the previous token will be invalidated.
By increasing the index tolerance, you will be able to generate tokens at different indexes so the tokens won’t get invalidated.

I will look into your request shortly.
Best,
Elisa

As we continue to integrate with Zoom APIs, we are starting to see an alarming increase in these authentication errors. We’re able to mitigate the issue for the moment by adding retry loops, but we are going to need this band-aid request approved ASAP to buy us the time until the underlying issue has been fixed.

Do you have an ETA on when we will have our block of index values added to our account?

Thanks!

Hi @david5
I looked into your issue and I was able to resolve it.
The Engineering team has increased your index tolerance to 5.
could you please confirm with me this is working now?
Thanks
Elisa

@elisa.zoom

Thanks for making the change. Unfortunately, I’m not seeing the expected behavior. Here’s what I’m doing:

  1. First I request an access token with query parameter index=0
curl -s -H 'Accept: application/json' -XPOST --user [clientId]:[clientSecret] 'https://zoom.us/oauth/token?grant_type=account_credentials&account_id=[accountId]&index=0' |jq -r .access_token > /tmp/zoom-oauth
  1. I then make a Zoom API request. For exmample:
curl -H "Authorization: Bearer $(cat /tmp/zoom-oauth)" -H 'Content-Type: application/json' https://api.zoom.us/v2/meetings/[meetingId]
  1. I then request another access token with a different index and throw it away
curl -s -H 'Accept: application/json' -XPOST --user [clientId]:[clientSecret] 'https://zoom.us/oauth/token?grant_type=account_credentials&account_id=[accountId]&index=1' |jq -r .access_token > /dev/null
  1. If I then make the same request as in step 2, I get the “Invalid access token” error.

If I understand how this was supposed to work, access tokens generated for different index values would not interfere with each other. Am I using the correct query parameter in my token requests?

Also, do you have a ticket or forum thread I could follow to track the progress for fixing this problem for real (i.e. not the hacky index work around)?

Thanks!

Hi @David,
I have replied to the support ticket and I will personally take it from there.
I totally understand how you are feeling about this and to ensure faster and more efficient communication I will reply there.

Best,
Elisa

The key rotation strategy I’ve adopted is to allocate two index values per environment and store a separate renewal time per access token that is much earlier (half the lifetime) than the expiration time for that token. The first process to notice the renewal time has elapsed will attempt to nudge forward the renewal time slightly into the future, and if it successfully did so (winner of the race condition), then it refreshes the access token for the index that isn’t currently in use, then swaps that access token and index into active use for the entire environment.

Thanks @MultiplayerSession , once I have the index stuff in place, that does seem like a reasonable work around until the underlying issue is fixed.

Hi @MultiplayerSession I just wanted to reach out to you to confirm that I am working with Engineering to debug this issue. The index tolerance has been increased for your app so I shared the information you shared with me with the team so they can investigate this further. Thanks for your patience.

@david5 This message was probably intended to ping you instead of me:

That said, I’m also seeing the same behavior where the index parameter doesn’t appear to be taking effect using similar replication steps, although I’m placing the request parameters in the request body instead of the query string:

Request URL: https://zoom.us/oauth/token
Request body: grant_type=account_credentials&account_id=***&index=2
Response body: {"access_token":"***","token_type":"bearer","expires_in":3599,"scope":"dashboard_meetings:read:admin dashboard_webinars:read:admin meeting:read:admin meeting:write:admin user:read:admin webinar:read:admin webinar:write:admin"}

I’m trying to troubleshoot with Zoom Developer Support in ticket number #14947692.

Update (15 September 2022): The correct parameter name should be token_index.

Thanks for catching this up! I probably got confused @MultiplayerSession
Also thanks for sharing the ticket that you have open with support… I will take a look into that ticket and I will reply in that thread

@elisa.zoom ,

Any movement on this? I’ve seen the updates through the support ticket and other channels, but things still aren’t working any differently. Any additional information needed from me to help debug the issue?

DAvid

Hi @david5
I do not have one at the moment, but I am actively looking into this