How to prevent HTTP 429 TooManyRequests errors with OCI Python SDK — #mytechretreat

4 min readJun 1, 2020

You probably ended up here because you have a script that makes many API calls to Oracle Cloud Infrastructure and some of your requests fail due to an HTTP 429 TooManyRequests error.

Why do you get those errors?

First of all, this is normal behavior. Oracle Cloud Infrastructure applies throttling to many API requests to prevent accidental or abusive use of resources. If you make too many requests too quickly, you might see some succeed and others fail.

API limiting, which is also known as rate limiting, is an essential component of Internet security, as DoS attacks can tank a server with unlimited API requests.

What to do in this case?

You should implement an exponential back-off strategy which will pause some requests when you hit the rate limit. There are many ways of achieving this, but let’s look at how to do this with the OCI Python SDK.

Retries

As you may have seen already, the OCI Python SDK does not use a retry strategy by default, but you can pass a retry_strategy keyword argument to every API call made via the SDK operations.

There are 3 possibilities when it comes to the retry strategy that can be passed to your operations:

The default retry strategy implemented in the SDK as DEFAULT_RETRY_STRATEGY
The NoneRetryStrategy which will not perform any retries
A custom retry strategy that can be built using the RetryStrategyBuilder

Break the OCI Rate Limiter

Before we implement the retry strategies, let’s just look at a simple example that will break the OCI API rate limiter so we can have something to build upon:

So, to explain the example — I am getting the list of subscribed regions 200 times using threads — the script runs from a Compute instance in OCI and I’m using Instance Principals for Authentication.

If you need help configuring the authentication check those two articles:

For authenticating via config file (run the script from a compute in OCI or from anywhere else), check this article
For authenticating via instance principals (run the script from a compute in OCI), check this article

This example will throw some HTTP 429 TooManyRequest errors that look something like this:

Exception in thread Thread-200:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "no_retry.py", line 50, in __get_regions
    self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id).data
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/identity/identity_client.py", line 6059, in list_region_subscriptions
    response_type="list[RegionSubscription]")
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 248, in call_api
    return self.request(request)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 363, in request
    self.raise_service_error(request, response)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 533, in raise_service_error
    original_request=request)
oci.exceptions.ServiceError: {'opc-request-id': 'XXXXXXXXXXXX', 'code': 'TooManyRequests', 'message': 'Too many requests for the user', 'status': 429}

Implementing the Retry Strategies

Let’s look at how we can implement the retry strategies in the example used earlier.

The Default Retry Strategy

Let’s look at the characteristics of the Default Retry Strategy:

5 total attempts
Total allowed elapsed time for all requests of 300 seconds (5 minutes)
Timeouts and connection errors
HTTP 429 (throttling)
HTTP 5xx (server errors)
Exponential backoff with jitter, using the base time to use in retry calculations will be 1 second, an exponent of 2. When calculating the next retry time we will raise this to the power of the number of attempts and maximum wait time between calls of 30 seconds

You can see the full list of characteristics here.

To implement this retry strategy, you must add a retry_strategy keyword argument to your API calls. Let’s do that to the API call used earlier to get the subscribed regions of my tenancy:

It is as simple as that!

Now you should be able to run the same script and you’ll not get any HTTP 429 TooManyRequests errors anymore.

This default retry strategy should work in most cases, but if you want to configure it, you can build your own retry strategy.

The Custom Retry Strategy

If we look in the source code of the retry strategy, we can find all the parameters that we can give to our custom retry strategy, all of them are optional and have default values:

Then, we just need to pass this retry strategy over to the API calls:

Conclusions

If you end up making many API calls against OCI, you may want to implement a retry strategy that can manage the HTTP 429 TooManyRequests errors. or timeouts or other types of errors.

We saw that there are two ways of doing this, either with a default retry strategy or by creating a custom retry strategy. This will help retry the requests that are “rejected” by the OCI API rate limiter, timeouts, server errors, etc, and make some pauses in between so you execute all the request successfully.

You can find on my GitHub all 3 complete examples showcased in this article: