Python Retry with Backoff — Core Concepts

Why Retry?

Network calls fail. Databases have momentary hiccups. APIs rate-limit you. Cloud services have brief outages. These failures are often transient — they resolve on their own within seconds or minutes.

A retry strategy lets your application recover from transient failures automatically, without human intervention. The key is retrying smartly.

Backoff Strategies

Fixed Interval

Wait the same amount of time between each retry:

• Retry 1: wait 2 seconds
• Retry 2: wait 2 seconds
• Retry 3: wait 2 seconds

Simple but can overwhelm a recovering service since all clients retry at the same pace.

Exponential Backoff

Double the wait time with each retry:

• Retry 1: wait 1 second
• Retry 2: wait 2 seconds
• Retry 3: wait 4 seconds
• Retry 4: wait 8 seconds

This dramatically reduces pressure on struggling services. After a few retries, clients are barely sending requests, giving the service maximum recovery time.

Exponential Backoff with Jitter

Add randomness to the wait time:

• Retry 1: wait 1 + random(0, 1) seconds
• Retry 2: wait 2 + random(0, 2) seconds
• Retry 3: wait 4 + random(0, 4) seconds

Without jitter, all clients that failed at the same time will retry at the same time, creating “retry storms.” Jitter spreads retries across a time window, smoothing out the load. AWS recommends this as their standard retry strategy.

What to Retry

Not all errors are worth retrying:

Retry these (transient failures):

• Connection timeouts
• HTTP 429 (Too Many Requests)
• HTTP 500, 502, 503, 504 (server errors)
• Database connection errors
• DNS resolution failures

Don’t retry these (permanent failures):

• HTTP 400 (Bad Request) — your data is wrong, sending it again won’t help
• HTTP 401/403 (Unauthorized/Forbidden) — credentials are wrong
• HTTP 404 (Not Found) — the resource doesn’t exist
• Validation errors

Retrying permanent failures wastes time and resources.

Retry Budgets

Unlimited retries are dangerous. A retry budget caps how much retry effort your application spends:

Max retries — Stop after N attempts (3-5 is typical). Each additional retry has diminishing returns.

Max elapsed time — Stop retrying after a total time window (e.g., 30 seconds). Even if you haven’t exhausted your retry count, the user shouldn’t wait forever.

Max backoff cap — Don’t let exponential backoff grow unbounded. Cap it (e.g., at 60 seconds). Otherwise, retry 10 would wait over 17 minutes.

The Retry Decision Flow

When a request fails:

1. Is the error retryable? If not, fail immediately.
2. Have we exceeded our retry budget? If yes, fail with the last error.
3. Should we wait? If the server sent a Retry-After header, use that timing.
4. Calculate backoff with jitter.
5. Wait and retry.

Common Misconception

“More retries are always better.” Beyond 3-5 retries, the probability of success drops sharply. If a service is down, retrying 20 times with exponential backoff means your request might wait over an hour. The user would have given up long ago. Set reasonable limits and fail fast when it’s clear the service isn’t recovering.

The one thing to remember: Combine exponential backoff with jitter and a strict retry budget — this gives transient failures time to resolve without overwhelming the failing service or making users wait forever.