Python Requests Retry: Backoff, Timeouts, and Error Handling

Retries make Python HTTP clients more resilient to brief network and server failures—but retrying the wrong request can duplicate an action, intensify a rate limit, or waste several minutes on an error that will not change.

The practical pattern is to mount an HTTPAdapter with an urllib3.util.Retry policy on a requests.Session, limit retries to suitable methods and failures, apply backoff, respect Retry-After, and set explicit timeouts.

Quick Answer

Python Requests does not retry failed connections by default. For controlled retries, pass an urllib3.util.Retry policy to Requests' HTTPAdapter and mount it on a Session. Retry bounded, safe requests after connection failures, 429, and selected 5xx responses. Set a timeout and exponential backoff, honor Retry-After, and do not automatically retry an unchanged 403 response.

total=4 means up to four retries after the initial request, for a maximum of five attempts within one adapter send to one URL. Requests handles redirects separately, so a redirected operation can make more than five network requests.

Complete Tested Python Requests Retry Example

Install Requests:

Then create a reusable retrying session:

This exact example was executed on September 22, 2026, with Python 3.11.2, Requests 2.34.2, and urllib3 2.8.0. A live request to https://example.com/ returned 200. Eleven local tests confirmed 503 recovery, Retry-After handling and its default cap, the special 413 behavior, 403 stopping, POST protection, exponential backoff with jitter, exhausted connection and read failures, redirect behavior, and exact attempt counts.

The policy uses raise_on_status=False so the final response is returned after the retry budget is exhausted. The following raise_for_status() call then raises a Requests HTTPError for the caller to handle. Exhausted connection failures raise ConnectionError rather than returning None, so the caller never attempts .status_code on a missing response. A direct timeout can raise a Requests timeout exception, while exhausted urllib3 read retries can reach Requests as ConnectionError wrapping a ReadTimeoutError; the example catches both paths.

Pass a complete URL, including http:// or https://. A value such as example.com is not a valid Requests URL because it has no scheme.

Key Takeaways

• Requests does not retry failed connections by default; configure retries explicitly through HTTPAdapter and urllib3.util.Retry.
• Retry transient failures, not every non-200 response. A conservative starting list is 429, 500, 502, 503, and 504; diagnose an unchanged 401, 403, 404, or 407 instead.
• Retry only methods whose repeat behavior you understand. The example limits status and read retries to GET, HEAD, and OPTIONS.
• Always set a timeout. Retries without timeouts can leave each attempt waiting indefinitely.
• Backoff reduces retry pressure; Retry-After communicates a server-requested minimum delay. Long delays are often better handled by a job scheduler than a sleeping worker.
• A proxy rotation policy and an HTTP retry policy solve different problems. Changing an IP is not a general fix for authorization errors or rate limits.

How Python Requests Retries Work

Three objects divide the work:

1. requests.Session reuses connection settings across requests.
2. HTTPAdapter connects the Session to urllib3's connection pools and accepts the retry policy.
3. urllib3.util.Retry defines retry counts, methods, status codes, backoff, and Retry-After handling.

Mounting the adapter on both prefixes matters:

If you mount only https://, an HTTP URL uses the default adapter and will not apply this policy. Mounting on a hostname prefix is also possible when only one service should receive the custom behavior.

Does Requests Retry by Default?

No. Requests' default HTTPAdapter uses max_retries=0. The official API documentation says failed connections are not retried by default and recommends passing an urllib3 Retry object when granular control is required.

This is different from using urllib3 directly, which has its own defaults. Do not infer Requests behavior from a standalone urllib3 example.

Retries vs. Total Attempts

Retry(total=4) permits up to four retries after the first adapter send to one URL. The maximum within that send is therefore:

total caps the combined retry budget and takes precedence over the individual connect, read, status, and other counters. Keeping other=0 prevents an unexpected error category from creating an unbounded retry path if the total policy is later changed.

This is not necessarily a five-request ceiling for the caller's entire session.get() operation. The Requests Session implementation processes redirects above the adapter layer, and every followed redirect can initiate another adapter send with its own retry history. Use allow_redirects=False when the operation must not follow redirects, or enforce a separate operation-level request/deadline budget when redirects are allowed.

Which HTTP Errors Should Python Requests Retry?

The correct response depends on both the failure and the HTTP method.

The example deliberately omits 408 because not every application treats it the same way. Add it only after confirming that repeating the relevant method is safe for the service.

Should Python Requests Retry a 403 Response?

Not automatically. A 403 Forbidden response means the server understood the request but refuses to fulfill it. Sending the same request again usually produces the same result.

Investigate the cause instead:

• Does the user or service account have permission?
• Is the authentication valid but missing a required scope?
• Does an allowlist, WAF rule, geographic policy, or resource policy deny the request?
• Has the site explicitly rejected this automated access?
• Is the response actually a block page rather than the expected content?

A later attempt can be reasonable after the permission, token, allowlist, request, or site-side rule has been corrected. That is a new condition—not a reason to place 403 in a generic automatic retry list. Changing proxies to repeatedly resubmit a forbidden request is not a sound retry strategy.

How Should Requests Handle 429 and Retry-After?

429 Too Many Requests is retryable only when the client also reduces pressure. respect_retry_after_header=True tells urllib3 to honor a valid Retry-After header for applicable responses.

Retry-After can contain either a number of seconds or an HTTP date. In urllib3 2.8.0, retry_after_max defaults to 21600 seconds—six hours. A larger header value is limited to that maximum; for example, a requested 24-hour delay becomes six hours inside the default policy. If an application cannot honor the server's complete requested wait, it should stop or defer the operation through a scheduler rather than retry early. A retry adapter controls one request chain; it does not replace a target-level rate limiter shared by all workers.

There is another important urllib3 rule: with respect_retry_after_header=True, an eligible method receiving 413, 429, or 503 with Retry-After can be retried even when that status is absent from status_forcelist. The primary example therefore may retry 413 only when the response supplies Retry-After; a 413 without that header is not in its configured status list and is returned immediately.

Do not respond to 429 by immediately changing IP addresses and continuing at the same aggregate rate. That evades the signal instead of correcting the load that caused it.

Exponential Backoff, Jitter, and Retry Budgets

Exponential backoff increases the delay as failures continue. In urllib3, backoff_factor controls the base calculation and backoff_max caps the computed delay. A retry can still wait longer when it honors a valid Retry-After value.

With a backoff_factor of 1, the first eligible retry can happen without a backoff delay; later consecutive failures produce delays on the order of 2, 4, and 8 seconds before jitter and caps. Treat that as an implementation detail of the installed urllib3 version rather than hard-coding a separate sleep schedule around the adapter.

The primary example sets backoff_jitter=0.5, which lets urllib3 add up to half a second of randomness to each computed backoff. This reduces the chance that many workers wake and retry simultaneously. The validated example targets urllib3 2.8.0; check compatibility before copying backoff_jitter into an older environment.

Backoff prevents rapid retry loops, but the total budget matters just as much. If five layers each make three attempts, one failed business operation can create hundreds of downstream requests. Define which layer owns retries and place one shared ceiling around the complete operation.

Timeouts and Exceptions

Requests does not time out unless a timeout is supplied. The tuple used in the primary example has two parts:

• 3.05 seconds is the connection timeout.
• 30 seconds is the read inactivity timeout.

The read value is not a deadline for the complete download. It limits how long Requests waits without receiving bytes from the socket. Use an outer deadline or task cancellation when the entire operation must finish within a fixed time.

Also note that adapter retries cannot reconstruct a partially consumed response body. A read failure that occurs after Requests has returned the response and begun streaming its body may escape the adapter's retry loop. If a complete download must be retried, handle it at the application layer, discard partial output, and repeat only when doing so is safe.

Catch the narrowest useful exception:

Avoid except Exception: pass. It hides exhausted retries, programming errors, certificate failures, and permanent responses while leaving the caller without a valid result.

This distinction follows the current Requests adapter implementation: a direct urllib3 ReadTimeoutError maps to Requests ReadTimeout, while an exhausted MaxRetryError that wraps a read timeout falls through to Requests ConnectionError.

Should You Retry POST, PUT, or DELETE Requests?

HTTP semantics distinguish safe and idempotent methods. GET, HEAD, and OPTIONS are safe. PUT and DELETE are defined as idempotent, but an application can still attach operational side effects to every request. POST is not generally idempotent.

RFC 9110 says a client should not automatically retry a non-idempotent request unless it knows the operation is idempotent or knows the original request was not applied.

The primary example therefore limits read and status retries to:

Retry POST only when the API explicitly supports it—for example, with a unique idempotency key that makes duplicate submissions return the original result. Confirm the API's exact guarantee before adding POST to allowed_methods.

One nuance: urllib3 can retry a connection-establishment failure even for another method because that class of failure is assumed to occur before the request reaches the server. Read and response-status retries are more dangerous because the request has already been sent.

When Should You Write Custom Retry Logic?

Use HTTPAdapter plus Retry for transport errors and clearly defined HTTP statuses. Add application-level logic when success depends on the body rather than the status—for example, an API returns 200 with an incomplete payload, a page is missing required fields, or a job returns a domain-specific “pending” state.

Do not casually wrap an already-retrying Session in another retry loop. The attempt counts multiply. If the adapter allows five total attempts and an outer loop runs four times, one operation can make 20 requests.

A custom layer should therefore:

• Share one total attempt or elapsed-time budget with the transport layer.
• Validate the returned content before recording success.
• Log the attempt number, elapsed time, exception or status, and final outcome.
• Stop immediately for authentication, authorization, validation, and malformed-request failures.
• Schedule long delays instead of making a worker sleep.
• Preserve idempotency for any operation that can change server state.

Python Requests Retries With Proxies

Retries and proxy rotation are separate controls. A retry decides whether to repeat an operation after a classified failure. A proxy policy decides which route and exit identity carry a request.

Use the failure layer to choose the response:

For scraping, one retrying Session can use an HTTP proxy through Requests' proxies mapping. Keep credentials in environment variables or a secret manager rather than source code. The HTTP proxy guide explains the protocol, while the backconnect proxy guide covers one-gateway pool rotation.

Provider-side rotating sessions suit independent jobs; sticky sessions suit stateful sequences. Do not rotate merely because a transient server error occurred. The IP rotation guide explains how rotation triggers differ from retry triggers, and the web scraping with proxies guide covers the wider collection system.

Common Python Requests Retry Problems

What Does “Max retries exceeded with url” Mean?

It means urllib3 exhausted the configured retry category or total budget. It does not identify one universal cause. Inspect the chained exception to distinguish DNS failure, refused connection, connect timeout, read timeout, TLS failure, proxy failure, or another transport problem. Confirm the complete URL, proxy configuration, and adapter mount before increasing the budget; additional attempts do not repair a permanent configuration error.

What Happens When Status Retries Are Exhausted?

With raise_on_status=False, urllib3 returns the final HTTP response after the retry budget. The primary example then calls raise_for_status(), producing a Requests HTTPError for the final 4xx or 5xx. With raise_on_status=True, urllib3 instead raises after exhausting a status retry. Choose one behavior deliberately and keep the caller consistent with it.

Why Can a Retrying Request Still Hang?

A retry count is not a time limit. Without timeout=, any individual attempt can wait indefinitely. Even with timeout=(connect, read), the tuple is not a deadline for the entire operation: redirects, retries, DNS resolution, repeated reads, and Retry-After delays can extend total elapsed time. Use an outer operation deadline or job cancellation when the whole task must finish within a fixed window.

How to Test a Retry Policy

Do not validate retry code by repeatedly calling an unreliable third-party website. Use a local or controlled endpoint that deliberately returns a known sequence.

Save the primary example as retry_example.py, then save this local-server test beside it as test_retry_example.py:

Run it with:

This verifies that:

1. Two temporary 503 responses followed by 200 produce exactly three attempts.
2. 429 honors a one-second Retry-After before succeeding.
3. A 24-hour Retry-After value is limited to urllib3's default maximum of 21600 seconds.
4. An eligible 413 carrying Retry-After is retried even though 413 is absent from status_forcelist.
5. 403 makes exactly one request.
6. A status-failing POST is not repeated by the conservative policy.
7. total=4 produces exactly five attempts for one non-redirected status or connection-failure path, including the initial request.
8. Exponential backoff with controlled jitter sleeps for 2.5 and 4.5 seconds in the tested four-attempt sequence.
9. An exhausted synthetic connection failure raises ConnectionError instead of returning None.
10. Exhausted header-read retries can surface as ConnectionError wrapping urllib3's ReadTimeoutError.
11. One redirect followed by five attempts at the destination produces six network requests, confirming that redirect handling sits outside one adapter-send budget.

Tests should count requests on the server side. Client logs alone can miss redirects, connection retries, and attempts performed inside the adapter.

A Reliable Retry Policy Is Selective

Use a Requests Session with HTTPAdapter and urllib3 Retry for bounded transport and status retries. Apply explicit timeouts, backoff, and Retry-After; repeat only understood methods and transient failures; validate successful bodies separately; and keep proxy rotation independent from retry decisions.