Hone logo
Hone
Problems

Robust API Client with Retry Logic

Many real-world applications interact with external APIs. These APIs can be unreliable, experiencing temporary outages, network issues, or rate limiting. This challenge asks you to implement a robust API client in Python that automatically retries failed requests, making your application more resilient to these common problems.

Problem Description

You are tasked with creating a function retry_request that attempts to make an HTTP request to a given URL. The function should implement retry logic with exponential backoff and jitter. If the initial request fails (e.g., due to a network error, HTTP status code 500, 503, or 429), the function should retry the request after a delay that increases exponentially with each attempt, but with a small random jitter added to avoid synchronized retries.

What needs to be achieved:

  • Create a function retry_request(url, max_retries, base_delay) that takes a URL, the maximum number of retries, and a base delay (in seconds) as input.
  • The function should make an HTTP GET request to the provided URL using the requests library.
  • If the request fails (raises an exception or returns an error status code), the function should retry the request.
  • The retry delay should increase exponentially with each attempt, starting from the base_delay.
  • A small random jitter (between 0 and base_delay / 2) should be added to the delay to avoid synchronized retries.
  • The function should return the response object if the request is successful after all retries.
  • If the maximum number of retries is reached without success, the function should raise an exception indicating the request failed.

Key Requirements:

  • Use the requests library for making HTTP requests.
  • Implement exponential backoff with jitter.
  • Handle exceptions gracefully.
  • Limit the number of retries.
  • Raise an exception if the request fails after all retries.

Expected Behavior:

  • If the initial request is successful, the function should return the response object immediately.
  • If the request fails, the function should retry according to the exponential backoff and jitter strategy.
  • The delay between retries should increase exponentially (e.g., base_delay, 2 * base_delay, 4 * base_delay, ...).
  • Jitter should be added to the delay to avoid synchronized retries.
  • If the request succeeds after any retry, the function should return the response object.
  • If the maximum number of retries is reached without success, the function should raise a RetryError exception.

Edge Cases to Consider:

  • Invalid URL format.
  • Network connectivity issues.
  • HTTP status codes indicating errors (e.g., 400, 401, 403, 404, 500, 502, 503).
  • Rate limiting (HTTP status code 429).
  • max_retries being zero.
  • base_delay being zero or negative.

Examples

Example 1:

Input: url="https://www.example.com", max_retries=3, base_delay=1
Output: <Response [200]>
Explanation: The request to example.com is successful on the first attempt.

Example 2:

Input: url="https://www.example.com/nonexistent", max_retries=3, base_delay=1
Output: Raises RetryError after 3 retries.
Explanation: The request to a nonexistent page fails repeatedly, and the function raises a RetryError after exhausting the retries.

Example 3: (Simulating a temporary error)

Input: url="https://httpstat.us/503", max_retries=2, base_delay=0.5
Output: <Response [200]> (after a retry)
Explanation: The URL returns a 503 Service Unavailable error initially. The function retries, and eventually the request succeeds.

Constraints

  • url must be a valid URL string.
  • max_retries must be a non-negative integer.
  • base_delay must be a positive float.
  • The exponential backoff delay should not exceed 60 seconds.
  • The jitter should be between 0 and base_delay / 2.
  • Use the requests library. Install it with pip install requests.

Notes

  • Consider using the time.sleep() function to introduce delays.
  • Use a RetryError exception to signal that the request failed after all retries. You can define this exception yourself.
  • The exponential backoff formula can be implemented as base_delay * (2 ** retry_count).
  • The jitter can be implemented using random.uniform(0, base_delay / 2).
  • Think about how to handle different types of exceptions that might occur during the request. You may want to retry only certain types of errors.
  • This problem focuses on the retry logic itself. Error handling beyond retries (e.g., logging, circuit breakers) is not required.
Loading editor...
python