Keep hitting the rate limit despite making way less calls than the limit

API and Webhooks
1

Hello all,

I am trying to pull all pass meetings in a period from a Dashboard api List Meetings and according to this page the rate limit is 16 requests/minute. I have slowed it down to 4 requests/minute and I keep getting back an error that I have reached the limit.

An entry from my log file to give you an idea of how often it is called:

2019-07-25 11:13:45,735 - etl - INFO - /metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=9ZwlUUZvoYZlR0LUPuuP4oe9LY8FUM2AFy42&type=past

2019-07-25 11:14:02,710 - etl - INFO - /metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=wtOlBzGd6NU86bFHkKxOJwX4GPn2qbeq6y43&type=past

2019-07-25 11:14:19,913 - etl - INFO - /metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=dJLtTBPEjWLzuMXeJsh16RSL8o9WMwNRON44&type=past

2019-07-25 11:14:37,049 - etl - INFO - metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=zMdGWMmALiaeoSR6RaRCVehM2665SWMUCJ45&type=past

2019-07-25 11:14:54,230 - etl - INFO - /metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=8w3kL9Y5JMeB3PzuO9fy6mnY6fX2DZRASS46&type=past

2019-07-25 11:15:11,442 - etl - INFO - /metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=SNCl6fBBIH0DuplCd3xXwi5B5bHEu7D2Gv47&type=past

2019-07-25 11:15:28,237 - etl - INFO -metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=fhf9rwIINYYoLLqWWIMg3LdazCJ5iiLgqa48&type=past

2019-07-25 11:15:44,966 - etl - INFO - metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=tp85VFbjR1xwDV2AhjSlemsJYjqatbnKie49&type=past

2019-07-25 11:16:02,127 - etl - INFO - /metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=nFwMy7ooNhUJD12G2FgHpSyn6fKLUu6Evj50&type=past

2019-07-25 11:16:19,433 - etl - INFO - metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=o52nmV1cn8FLzWYAEsfRWVqHDXvd3XIr0P51&type=past

2019-07-25 11:16:36,253 - etl - INFO -/metrics/meetings?page_size=300&from=2019-03-01&to=2019-07-01&next_page_token=tvHCalQMQxk5Y93MxERTZDLgPH4q4OL3PG52&type=past

2019-07-25 11:16:36,634 - etl - INFO - 429

2019-07-25 11:16:36,634 - etl - INFO - {‘code’: 403, ‘message’: ‘Sorry, the maximum number of metrics api requests every minute are already reached. Please try later.’}

I am working on this ETL so I am making the calls manually and I guarantee I have not reached the other limit of 32000 max requests per day.

Can anyone shed any light on what is going on? This would make it very difficult to configure in production and let it run and would also cause more calls towards your API because every time it gets dropped it will have to start again because the pagination token would have expired in the meantime. Does not sound productive for anyone involved.

Thanks a lot for your help.

Rali

P.S Had to remove the beginning of the url from the log or it would not let me post this :thinking:

3

Hi @ralitsa.ivanova,

Thanks for reaching out. To further elaborate our rate limits 16 request/min means 1 request per 3.75 seconds. A good workaround for you would be to make 1 request every 10 seconds. Let us know if that works for you.

Thanks

4

Hello Michael,

thank you for your response.

If the rate limit you are suggesting is 1 request per 10 seconds, how come the official one is one request every 3.75? This is a large volume of data and the request you are suggesting is significant.

However, as you can see from the datetime value in the logs that I have attached, there is one request every 15 seconds, which is even lower than what you are suggesting and it still hits the rate limit.

Any ideas?

Thank you very much for your help.

Best,
Rali

5

Hi @ralitsa.ivanova,

Would you be able to provide us your developer email and app ID so that we can take a closer look at your rate limit issue?

Thanks

6

Hi @michael_p.zoom,

thanks for your response and sorry for the delay.

The developer email is techops-incubator@thoughtworks.com and the app id should be FNQsICFScG0dm1D0R3Ts. I am accessing all the meetings historically, so it is a large chunk of data and the API limit is getting in the way.

Thank you for your help.

Best,
Rali

7

Thanks @ralitsa.ivanova,

We are looking into this and will post back with updates!

-Tommy

8

Hi @ralitsa.ivanova,

Looks like your request were hitting the same servers instead of different ones, that’s why the rate limits were happening. Our team is currently working on improvements for rate limits to avoid situations like these in a future API release.
Right now the best options would be to reduce the frequency API calls. Let us know if that helps, if not then we can you in contact with our support team to increase the rate limit.

Thanks

1 Like