Randomly receiving "Invalid access token" for Server-to-Server OAuth

Hi @David

Thank you for sharing more details with me.
So yes if each of your instances requests an access token with the same credentials (same Server-to-Server OAuth app) then the previous token will be invalidated.
By increasing the index tolerance, you will be able to generate tokens at different indexes so the tokens won’t get invalidated.

I will look into your request shortly.
Best,
Elisa

As we continue to integrate with Zoom APIs, we are starting to see an alarming increase in these authentication errors. We’re able to mitigate the issue for the moment by adding retry loops, but we are going to need this band-aid request approved ASAP to buy us the time until the underlying issue has been fixed.

Do you have an ETA on when we will have our block of index values added to our account?

Thanks!

Hi @david5
I looked into your issue and I was able to resolve it.
The Engineering team has increased your index tolerance to 5.
could you please confirm with me this is working now?
Thanks
Elisa

@elisa.zoom

Thanks for making the change. Unfortunately, I’m not seeing the expected behavior. Here’s what I’m doing:

  1. First I request an access token with query parameter index=0
curl -s -H 'Accept: application/json' -XPOST --user [clientId]:[clientSecret] 'https://zoom.us/oauth/token?grant_type=account_credentials&account_id=[accountId]&index=0' |jq -r .access_token > /tmp/zoom-oauth
  1. I then make a Zoom API request. For exmample:
curl -H "Authorization: Bearer $(cat /tmp/zoom-oauth)" -H 'Content-Type: application/json' https://api.zoom.us/v2/meetings/[meetingId]
  1. I then request another access token with a different index and throw it away
curl -s -H 'Accept: application/json' -XPOST --user [clientId]:[clientSecret] 'https://zoom.us/oauth/token?grant_type=account_credentials&account_id=[accountId]&index=1' |jq -r .access_token > /dev/null
  1. If I then make the same request as in step 2, I get the “Invalid access token” error.

If I understand how this was supposed to work, access tokens generated for different index values would not interfere with each other. Am I using the correct query parameter in my token requests?

Also, do you have a ticket or forum thread I could follow to track the progress for fixing this problem for real (i.e. not the hacky index work around)?

Thanks!

Hi @David,
I have replied to the support ticket and I will personally take it from there.
I totally understand how you are feeling about this and to ensure faster and more efficient communication I will reply there.

Best,
Elisa

The key rotation strategy I’ve adopted is to allocate two index values per environment and store a separate renewal time per access token that is much earlier (half the lifetime) than the expiration time for that token. The first process to notice the renewal time has elapsed will attempt to nudge forward the renewal time slightly into the future, and if it successfully did so (winner of the race condition), then it refreshes the access token for the index that isn’t currently in use, then swaps that access token and index into active use for the entire environment.

Thanks @MultiplayerSession , once I have the index stuff in place, that does seem like a reasonable work around until the underlying issue is fixed.

Hi @MultiplayerSession I just wanted to reach out to you to confirm that I am working with Engineering to debug this issue. The index tolerance has been increased for your app so I shared the information you shared with me with the team so they can investigate this further. Thanks for your patience.

@david5 This message was probably intended to ping you instead of me:

That said, I’m also seeing the same behavior where the index parameter doesn’t appear to be taking effect using similar replication steps, although I’m placing the request parameters in the request body instead of the query string:

Request URL: https://zoom.us/oauth/token
Request body: grant_type=account_credentials&account_id=***&index=2
Response body: {"access_token":"***","token_type":"bearer","expires_in":3599,"scope":"dashboard_meetings:read:admin dashboard_webinars:read:admin meeting:read:admin meeting:write:admin user:read:admin webinar:read:admin webinar:write:admin"}

I’m trying to troubleshoot with Zoom Developer Support in ticket number #14947692.

Update (15 September 2022): The correct parameter name should be token_index.

Thanks for catching this up! I probably got confused @MultiplayerSession
Also thanks for sharing the ticket that you have open with support… I will take a look into that ticket and I will reply in that thread

@elisa.zoom ,

Any movement on this? I’ve seen the updates through the support ticket and other channels, but things still aren’t working any differently. Any additional information needed from me to help debug the issue?

DAvid

Hi @david5
I do not have one at the moment, but I am actively looking into this

For posterity and anyone else who happens to run across this thread while scratching their head about how Server-to-Server OAuth works:

  1. Requesting a new Server-to-Server OAuth token invalidates the previous one.
  2. Zoom can increase the “index” count for your account, allowing you to request different tokens with different indexes that do not interfere with each other (i.e. requesting a token for index 1 doesn’t invalidate the token requested for index 2).
  3. The token index is specified in the call to https://zoom.us/oauth/token by passing the token_index query parameter (https://zoom.us/oauth/token?token_index=2)

This should allow folks to build out a background oauth token rotation strategy where the token is updated in a central place, using a new index value (e.g. database, AWS SSM or secrets manager, etc.) allowing applications to continue using the previous token until they can read the new value from the central location.

5 Likes

Hello @elisa.zoom ,

We are having the same issues, can you help please?
I created this ticket: https://support.zoom.us/hc/en-us/requests/16360848

Hello,
I am having the same issue and would the like the index for our server to server app to be increased to 6.
I had opened this ticket https://support.zoom.us/hc/en-us/requests/16423969 but was asked to post to the dev forum first. Please let me know how long this will take to resolve.
Also wanted to know if there is limit to the number of server to server oauth apps I can have on our account?
Sumit

I can’t help with increasing your limit, but I can say that the index is the limit to the number of oauth tokens you can have active at one time. We had to build a token rotation service that would flip between two indexes. We allocated some indexes to our dev environment and some to our prod.

Developer Support limited me to 3 concurrent index values. If you need more than that, my recommendation is to create multiple identical Server-to-Server OAuth applications corresponding to the number of concurrent tokens you need (each application represents 1 index value).

I was hoping to get 2 tokens per environment (3 environments). I thought I read in this post that someone’s index tolerance was increased to 5.
@david5 , can you share some details on the service you created?

@onemancat We run in AWS, so we wrote an AWS lambda function that is triggered on a schedule (every 50 minutes). The function updates a secret stored in AWS Secrets Manager. We store the oauth token, the index used to generate the token and the timestamp the token was generated. The Lambda function increments the index (circularly within a range), generates a new token and updates Secrets Manager.

The services that need to make Zoom API calls look up the Secrets Manager value and use the token within to make the API calls. They also use the timestamp value to determine when they should go back to Secrets Manager to look for a new token.

Thanks so much. That makes sense.