Python Idempotency Patterns — Core Concepts

What Is Idempotency?

An operation is idempotent if performing it multiple times produces the same result as performing it once. In API design, this means a client can safely retry a request without causing unintended side effects.

Some HTTP methods are naturally idempotent:

• GET — Fetching data doesn’t change anything. Always idempotent.
• PUT — “Set this resource to this state.” Doing it twice gives the same state.
• DELETE — Deleting something twice? The second time, it’s already gone. Same outcome.
• POST — Not naturally idempotent. Sending the same POST twice might create two records.

The challenge is making POST endpoints (create operations, payments, submissions) safe to retry.

Idempotency Keys

The most common solution: the client sends a unique identifier with each request. The server uses this key to detect duplicates.

How it works:

1. Client generates a unique key (usually a UUID) before making the request
2. Client includes this key in the request header: Idempotency-Key: abc-123
3. Server processes the request and stores the result alongside the key
4. If the same key arrives again, the server returns the stored result without reprocessing

Stripe popularized this pattern. Every payment API call can include an idempotency key, making retries completely safe.

Database-Level Protection

Even without explicit idempotency keys, you can protect against duplicates using database constraints:

Unique constraints prevent inserting the same record twice. If a user submits an order and the request retries, a unique constraint on (user_id, order_reference) rejects the duplicate insert.

Upsert operations (INSERT … ON CONFLICT) combine insertion with conflict handling. Instead of failing on a duplicate, the database updates the existing record or does nothing.

These work well for operations that have a natural unique identifier — like an order number, a transaction reference, or an email address.

State Machine Approach

For operations that go through stages (pending → processing → completed), model the workflow as a state machine:

1. Create a record in “pending” state
2. Process the operation
3. Move to “completed” state

If the request retries during step 2, the system sees the record already exists and is in “processing.” It can either wait for the original to finish or return early.

This pattern prevents the classic double-payment scenario: the payment record transitions through states, and each transition is validated against the current state.

Where Idempotency Matters Most

Payment processing — The highest-stakes scenario. Double charges destroy user trust.

Email/notification sending — Sending the same welcome email three times is annoying. Track which notifications have been sent.

Resource creation — Creating duplicate database records causes data integrity issues downstream.

Webhook handling — Webhook providers often retry delivery. Your handler must handle the same webhook arriving multiple times.

Queue processing — Message queues deliver “at least once.” Your consumer must handle duplicate messages.

Common Misconception

“Idempotency is only for payment APIs.” Every API that creates resources or triggers side effects benefits from idempotency. A webhook handler that processes the same event twice, a form submission that creates duplicate tickets, or a sync job that re-imports the same data — these are all idempotency problems. The payment case is just the one where the consequences are most visible.

The one thing to remember: Make POST endpoints idempotent with client-provided keys, database constraints, or state machines — your API should produce the same result whether a request arrives once or five times.