Introduction
Network failures, timeouts, and client retries can cause the same operation to be executed multiple times. Without proper idempotency handling, this can lead to duplicate charges, duplicated records, or inconsistent state. Idempotency ensures that executing an operation multiple times produces the same result as executing it once.
Why Idempotency Matters in Production
In production systems, idempotency failures translate directly to revenue loss and data corruption. The
canonical example is payment processing: a network timeout causes your API client to retry a charge request,
and without idempotency, the customer gets charged twice. This exact scenario is why Stripe requires clients
to send an Idempotency-Key header — Stripe deduplicates requests with the same key, guaranteeing
at-most-once processing. Beyond payments, non-idempotent operations cause race conditions in inventory systems
(deducting stock twice), duplicate user registrations (same email, two accounts), and inconsistent database
state from retried writes. The critical design decision is distinguishing operations that are naturally
idempotent — GET, DELETE, PUT with full payloads — from those requiring explicit idempotency keys, such
as payment intents, order creation, and any operation with side effects you cannot safely replay.
Understanding Idempotency
Before diving into implementation, it is essential to build a precise mental model of what idempotency means at the HTTP layer and why some operations are naturally idempotent while others require explicit machinery. An operation is idempotent when repeating it with the same input produces the same observable state as executing it exactly once. This property is independent of the response returned: the first attempt may return a freshly created resource while a retry returns the same resource, but the system state remains identical either way.
In the HTTP specification, methods have well-defined idempotency semantics that map cleanly onto real-world use cases. GET requests are inherently safe and idempotent because they only read state. PUT is idempotent because it replaces a resource with a full, deterministic payload — replaying it converges to the same final state. DELETE is idempotent from a state perspective: once a resource is gone, deleting it again leaves the system unchanged, even if a second call returns a different status code. The problematic method is POST, which by definition creates a new resource each time it is invoked. This is precisely why payment, order, and transfer endpoints — all built on POST — require an explicit idempotency contract layered on top of the HTTP semantics.
The diagrams below illustrate both sides of the problem. The first summarizes the classification of idempotent versus non-idempotent operations. The second walks through the classic duplicate-payment failure mode, showing how a client timeout followed by a retry can cause a payment to be processed twice at the database layer. Together they motivate every technique covered in the rest of this article.
┌─────────────────────────────────────────────────────────────────┐
│ Idempotency Concepts │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Idempotent Operations: │
│ │
│ GET /users/123 → Always returns user 123 │
│ DELETE /users/123 → Deleting user 123 twice = same as once │
│ PUT /users/123 → Setting name to "John" (same value) │
│ │
│ Non-Idempotent Operations: │
│ │
│ POST /orders → Creating order (each call = new order) │
│ POST /payments → Each call = new payment │
│ DELETE /items/1 → First call deletes, second returns 404 │
│ │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ The Problem: Duplicate Payments │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Client Server Database │
│ │ │ │ │
│ │ POST /pay │ │ │
│ │──────────────────▶│ │ │
│ │ │ (processing) │ │
│ │ (timeout) │ │ │
│ │✗ ───────────────▶│ │ │
│ │ │ │ │
│ │ (retry) │ │ │
│ │ POST /pay │ │ │
│ │──────────────────▶│ │ │
│ │ │ Process payment │ │
│ │ │──────────────────▶│ │
│ │ │ │ │
│ │ │ Process payment │ │
│ │ │──────────────────▶│ (DUPLICATE!) │
│ │ │ │ │
│ │◀─────────────────│ │ │
│ │ 200 OK │ │ │
│ │
│ Solution: Idempotency Key │
│ │
└─────────────────────────────────────────────────────────────────┘
Idempotency Keys
The core mechanism for making POST endpoints safe under retries is the idempotency key: a client-generated
unique identifier sent in a request header that lets the server recognize and deduplicate retries. When a
client sends a request with an Idempotency-Key, the server records the key along with the response it
produced. If the client retries — for example after a timeout or network blip — it sends the same key, and the
server recognizes it, short-circuits the business logic, and returns the stored response instead of executing
the operation a second time.
The design choices in the code below reflect production reality. First, the IdempotencyRecord stores not
just a boolean “processed” flag but the full response status and body. This is critical because retries must
return exactly what the first attempt would have returned — replaying the original response keeps client logic
simple and preserves observable behavior. Second, records carry an expiry timestamp so the store can bound its
memory footprint; a default TTL of 24 hours matches the practical window in which clients retry. Third, all
access to the in-memory store is guarded by a reentrant lock, since HTTP servers handle many concurrent
requests and a race here would reintroduce exactly the duplicate-processing bug we are trying to eliminate.
The IdempotentEndpoint decorator ties the pieces together at the HTTP boundary. It enforces that a key is
present and well-formed (rejecting keys shorter than 16 characters early), checks for an existing record, and
only executes the underlying handler when no prior result exists. Notice that failures are deliberately not
cached: if the underlying function raises, the wrapper re-raises without storing anything, so a subsequent
retry with the same key gets a fresh chance to succeed. The X-Idempotent-Replayed response header also gives
clients a way to distinguish a genuine first execution from a replay, which is invaluable for debugging.
import uuid
import hashlib
import time
from dataclasses import dataclass
from typing import Optional
import threading
@dataclass
class IdempotencyRecord:
"""Record of an idempotent request."""
key: str
response_status: int
response_body: dict
created_at: float
expires_at: float
class IdempotencyStore:
"""Store for idempotency records."""
def __init__(self, ttl_seconds: int = 86400):
self.records = {}
self.lock = threading.RLock()
self.ttl = ttl_seconds
def get(self, key: str) -> Optional[IdempotencyRecord]:
"""Get idempotency record if exists and valid."""
with self.lock:
record = self.records.get(key)
if record and time.time() < record.expires_at:
return record
# Clean up expired
if record:
del self.records[key]
return None
def set(self, key: str, status: int, body: dict):
"""Store idempotency record."""
with self.lock:
now = time.time()
self.records[key] = IdempotencyRecord(
key=key,
response_status=status,
response_body=body,
created_at=now,
expires_at=now + self.ttl
)
def delete(self, key: str):
"""Delete idempotency record."""
with self.lock:
self.records.pop(key, None)
def cleanup_expired(self):
"""Clean up expired records."""
with self.lock:
now = time.time()
expired = [
k for k, v in self.records.items()
if now >= v.expires_at
]
for k in expired:
del self.records[k]
class IdempotentEndpoint:
"""Decorator for idempotent endpoints."""
def __init__(self, store: IdempotencyStore):
self.store = store
def __call__(self, func):
"""Decorator to make endpoint idempotent."""
def wrapper(request, *args, **kwargs):
# Get idempotency key from header
idempotency_key = request.headers.get('Idempotency-Key')
if not idempotency_key:
return {"error": "Idempotency-Key required"}, 400
# Validate key format
if len(idempotency_key) < 16:
return {"error": "Invalid idempotency key"}, 400
# Check for existing request
existing = self.store.get(idempotency_key)
if existing:
return (
existing.response_body,
existing.response_status,
{'X-Idempotent-Replayed': 'true'}
)
# Execute request
try:
result = func(request, *args, **kwargs)
# Store successful result
if isinstance(result, tuple):
body, status = result
else:
body, status = result, 200
self.store.set(idempotency_key, status, body)
return body, status
except Exception as e:
# Don't store failed results
raise
return wrapper
This in-memory implementation is instructive but has important limitations in production. A single-process dictionary cannot survive restarts, cannot be shared across multiple application instances, and will lose all idempotency records if the server crashes mid-request. For that reason, real systems store idempotency records in the same durable store that holds the business data, or in a shared cache such as Redis that survives individual node failures. The persistence guarantees of the idempotency store define how strong your at-most-once guarantee is: a store that can lose records degrades to at-least-once behavior, which is why the database-backed approach covered next is the more common production pattern.
Database-Level Idempotency
Application-layer idempotency protects a single service, but the strongest guarantees come when the idempotency check and the business write share the same transactional boundary. If a payment is processed and then the process crashes before the idempotency record is written, a retry will process the payment a second time. The database-backed approach below eliminates that window by performing the lookup, the business operation, and the record insert inside the same connection and committing them together, so either all succeed or none do.
The execute_idempotent method implements this pattern: it first queries the idempotency_keys table for an
existing result, and only if none exists runs the provided operation. Because everything happens on a single
connection, the SELECT and the subsequent INSERT observe a consistent view of the database. A unique primary
key on the key column provides a second line of defense: even if two concurrent requests race and both miss
the initial lookup, the second INSERT fails with an IntegrityError, which the
execute_with_unique_constraint method converts into a fetch of the already-created record.
This approach also demonstrates a clean separation of concerns. The IdempotentDatabase class owns the
mechanics of idempotency — checking keys, storing results, handling races — while the PaymentRepository only
knows how to construct payment-specific operations. The operation closure returned by do_payment captures
the incoming request data, letting the generic idempotency machinery call arbitrary business logic without
needing to understand it. The trade-off is subtle but important: because the result is serialized to JSON and
stored in the same transaction, response bodies must be JSON-serializable, and large payloads can bloat the
idempotency table over time unless records are pruned.
import sqlite3
from typing import Optional
import json
import uuid
class IdempotentDatabase:
"""Database operations with idempotency support."""
def __init__(self, db_path: str):
self.conn = sqlite3.connect(db_path)
self._init_tables()
def _init_tables(self):
"""Initialize idempotency table."""
self.conn.execute("""
CREATE TABLE IF NOT EXISTS idempotency_keys (
key TEXT PRIMARY KEY,
result TEXT,
status_code INTEGER,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
self.conn.commit()
def execute_idempotent(self, key: str, operation: callable) -> dict:
"""Execute operation with idempotency check."""
cursor = self.conn.cursor()
# Check for existing result
cursor.execute(
"SELECT result, status_code FROM idempotency_keys WHERE key = ?",
(key,)
)
row = cursor.fetchone()
if row:
return json.loads(row[0]), row[1]
# Execute operation
try:
result = operation()
self.conn.commit()
# Store result
self.conn.execute(
"INSERT INTO idempotency_keys (key, result, status_code) VALUES (?, ?, ?)",
(key, json.dumps(result[0]), result[1])
)
self.conn.commit()
return result
except Exception as e:
self.conn.rollback()
raise
def execute_with_unique_constraint(self, table: str,
unique_fields: list[str],
data: dict) -> int:
"""Execute insert with unique constraint handling."""
unique_cols = ', '.join(unique_fields)
placeholders = ', '.join(['?'] * len(unique_fields))
values = tuple(data.get(f) for f in unique_fields)
try:
cursor = self.conn.cursor()
# Try to insert
cols = ', '.join(data.keys())
val_placeholders = ', '.join(['?'] * len(data))
cursor.execute(
f"INSERT INTO {table} ({cols}) VALUES ({val_placeholders})",
tuple(data.values())
)
self.conn.commit()
return cursor.lastrowid
except sqlite3.IntegrityError as e:
# Duplicate - fetch existing record
cursor.execute(
f"SELECT id FROM {table} WHERE {unique_cols} = ?",
values
)
row = cursor.fetchone()
return row[0] if row else None
class PaymentRepository:
"""Payment repository with idempotency."""
def __init__(self, db: IdempotentDatabase):
self.db = db
def create_payment(self, payment_data: dict) -> dict:
"""Create payment with idempotency."""
key = payment_data.get("idempotency_key")
def do_payment():
return self._process_payment(payment_data)
return self.db.execute_idempotent(key, do_payment)
def _process_payment(self, data: dict) -> dict:
"""Actual payment processing logic."""
# Payment processing logic here
return {
"payment_id": str(uuid.uuid4()),
"status": "completed",
"amount": data["amount"]
}, 201
Both approaches so far solve the same problem in different places: application code in memory versus the database transaction. The in-memory store is fast and simple but non-durable; the database-backed store is durable and race-safe but couples idempotency records to the business database. A hybrid pattern — writing the idempotency record in Redis first with a short TTL and falling back to the database for the final commit — combines the best of both and is the architecture used by many real payment platforms. Whichever you choose, the invariant to preserve is that the idempotency record and the side effect it represents must be written atomically with respect to each other.
Optimistic Locking
Idempotency keys protect against duplicate submission, but they do not solve a related problem: concurrent or conflicting updates to the same resource. When two clients read the same row, both modify it, and both write back, the last writer silently overwrites the first writer’s changes. Optimistic locking addresses this by versioning each resource and refusing writes that are based on a stale version.
The pattern is simple to reason about. Each entity carries an integer version that increments on every
successful update. When a client reads a resource, it receives the current version, typically surfaced as an
HTTP ETag header. When the client attempts an update, it sends the version back in an If-Match header. The
server compares the expected version against the current one: if they match, the update proceeds and the
version advances; if they differ, the update is rejected with a 409 Conflict, signaling the client to re-read
the resource and reconcile. This is exactly how many REST APIs implement last-writer-wins prevention without
holding locks.
The code below shows both the optimistic-lock store and the REST-layer glue around it. The store-level
update method compares the stored entity’s version with the caller’s expected version and raises
OptimisticLockError on mismatch, while the VersionedAPI translates that error into the HTTP 409 response.
The ETag format chosen — "<entity-id>-<version>" — is deliberate: it is stable, cheap to compute, and easily
parsed back into a version integer on the server side. Note also the graceful 400 response when the If-Match
header is entirely absent, which keeps the API honest about requiring optimistic concurrency rather than
silently falling back to unsafe blind writes.
import threading
from dataclasses import dataclass
from typing import Optional
@dataclass
class VersionedEntity:
id: str
version: int
data: dict
class OptimisticLockStore:
"""Store with optimistic locking."""
def __init__(self):
self.entities = {}
self.lock = threading.RLock()
def get(self, entity_id: str) -> Optional[VersionedEntity]:
with self.lock:
return self.entities.get(entity_id)
def update(self, entity_id: str, expected_version: int,
new_data: dict) -> VersionedEntity:
"""Update with optimistic locking."""
with self.lock:
entity = self.entities.get(entity_id)
if not entity:
# Create new entity
entity = VersionedEntity(
id=entity_id,
version=1,
data=new_data
)
self.entities[entity_id] = entity
return entity
# Check version
if entity.version != expected_version:
raise OptimisticLockError(
f"Version mismatch: expected {expected_version}, "
f"found {entity.version}"
)
# Update
entity.version += 1
entity.data = new_data
return entity
class OptimisticLockError(Exception):
"""Exception for optimistic lock failures."""
pass
# REST API Implementation
class VersionedAPI:
"""API with optimistic locking via ETag."""
def __init__(self, store: OptimisticLockStore):
self.store = store
def get_with_etag(self, entity_id: str) -> tuple[dict, str, int]:
"""Get entity with ETag."""
entity = self.store.get(entity_id)
if not entity:
return {"error": "Not found"}, 404, None
# Generate ETag
etag = f'"{entity.id}-{entity.version}"'
return entity.data, 200, etag
def update(self, entity_id: str, data: dict,
if_match: str) -> tuple[dict, int]:
"""Update with optimistic locking."""
if not if_match:
return {"error": "If-Match header required"}, 400
# Parse ETag
try:
expected_version = int(if_match.strip('"').split('-')[1])
except (ValueError, IndexError):
return {"error": "Invalid ETag"}, 400
try:
entity = self.store.update(entity_id, expected_version, data)
etag = f'"{entity.id}-{entity.version}"'
return {
"data": entity.data,
"version": entity.version
}, 200
except OptimisticLockError as e:
return {"error": str(e)}, 409
# Example: ETag in response headers
def handle_get(request, entity_id: str):
api = VersionedAPI(store)
data, status, etag = api.get_with_etag(entity_id)
if status == 200:
return data, 200, {'ETag': etag}
return data, status
def handle_update(request, entity_id: str, data: dict):
api = VersionedAPI(store)
if_match = request.headers.get('If-Match')
result, status = api.update(entity_id, data, if_match)
if status == 200:
result['ETag'] = f'"{entity_id}-{result["version"]}"'
return result, status
Optimistic locking trades a small amount of client complexity for a large gain in correctness. The client must handle the 409 Conflict case and re-fetch, but in exchange the server never needs to hold locks, never risks deadlock, and stays trivially horizontally scalable because state is only touched at the moment of the write. When combined with idempotency keys, the two techniques compose cleanly: the key prevents duplicate side effects from retries, while the version guard prevents conflicting concurrent updates from silently overwriting each other. Together they form the foundation of a safe mutation API.
Client-Side Idempotency
So far every mechanism has lived on the server, but idempotency is a contract between both parties, and the
client has just as much responsibility to get right. The IdempotentClient below shows what a well-behaved
client looks like. It automatically attaches a unique Idempotency-Key header to every request, freeing
callers from remembering to generate one, while still allowing an explicit key to be passed for operations
that must be retry-safe, such as payments.
The retry loop is where the client earns its keep. On a timeout, the client cannot know whether the server actually processed the request before the connection died — the classic unknown-state problem in distributed systems. The safe answer is to retry with the same key, which the client does automatically with exponential backoff. A 409 status is treated specially: it signals that another in-flight request already holds the key, so the client backs off and retries rather than treating it as a hard error. This coordination prevents two concurrent attempts from double-charging a customer.
There is an important asymmetry to appreciate here. The client can retry aggressively with exponential backoff because the server guarantees deduplication; without that server-side contract, retrying would be dangerous. Conversely, the server relies on the client to keep the key stable across retries. The same key must be reused for every attempt of the same logical operation, and a different key must be used for a genuinely new operation. Many subtle production bugs trace back to clients regenerating a fresh key on each retry, silently breaking the very guarantee they are relying on.
import requests
import uuid
import time
from typing import Callable
import threading
class IdempotentClient:
"""HTTP client with automatic idempotency."""
def __init__(self, base_url: str = "", retry_on_timeout: bool = True):
self.base_url = base_url
self.retry_on_timeout = retry_on_timeout
self.pending_requests = {}
self.lock = threading.Lock()
def request(self, method: str, endpoint: str,
idempotency_key: str = None,
**kwargs) -> requests.Response:
"""Make request with idempotency support."""
url = f"{self.base_url}{endpoint}"
# Generate idempotency key if not provided
if not idempotency_key:
idempotency_key = str(uuid.uuid4())
headers = kwargs.pop('headers', {})
headers['Idempotency-Key'] = idempotency_key
max_retries = 3
last_error = None
for attempt in range(max_retries):
try:
response = requests.request(
method,
url,
headers=headers,
**kwargs
)
# Check if response indicates in-progress
if response.status_code == 409:
# Another request in progress, wait and retry
time.sleep(0.5 * (attempt + 1))
continue
return response
except requests.Timeout as e:
last_error = e
if not self.retry_on_timeout:
raise
# Check if request might have succeeded
if attempt < max_retries - 1:
print(f"Request timeout, retrying ({attempt + 1}/{max_retries})")
time.sleep(2 ** attempt) # Exponential backoff
continue
raise last_error
raise last_error
def post(self, endpoint: str, data: dict, **kwargs) -> requests.Response:
return self.request('POST', endpoint, json=data, **kwargs)
def put(self, endpoint: str, data: dict, **kwargs) -> requests.Response:
return self.request('PUT', endpoint, json=data, **kwargs)
def delete(self, endpoint: str, **kwargs) -> requests.Response:
return self.request('DELETE', endpoint, **kwargs)
# Usage
client = IdempotentClient("https://api.example.com")
# Payment request - use same key for retries
payment_data = {
"amount": 100.00,
"currency": "USD",
"customer_id": "cust_123"
}
# First attempt
try:
response = client.post(
"/payments",
payment_data,
idempotency_key="payment_cust123_12345"
)
except requests.Timeout:
# Retry with same key - won't create duplicate
response = client.post(
"/payments",
payment_data,
idempotency_key="payment_cust123_12345"
)
The client-side patterns here generalize well beyond payments. Any long-running, state-changing operation benefits from the same discipline: generate a stable key once, reuse it across retries, back off on transient failures, and treat unknown outcomes as retryable. When the server and client both honor the contract, the system as a whole behaves as at-most-once from the user’s perspective even though individual components are at-least-once. This layering of guarantees is a recurring theme in reliable distributed systems, and it reappears in the message-queue context covered next.
Idempotency in Message Queues
HTTP APIs are not the only place duplicate executions occur; message queues are arguably even more prone to them. Distributed message brokers such as Kafka, RabbitMQ, and SQS provide at-least-once delivery semantics by design: a consumer that crashes after processing a message but before acknowledging it will receive that message again on reconnection. The result is the same failure mode as a retried payment — a side effect executed twice — but at the level of background jobs, data pipelines, and event handlers.
The solution mirrors the idempotency-key approach, adapted to the message’s content. The MessageIdempotency
class derives a deterministic fingerprint from the message’s stable fields — type, id, timestamp, and source —
hashes it with SHA-256, and uses it as a key in Redis. Before processing, the consumer checks whether the key
already exists; after processing, it records the key with a 24-hour TTL. If a duplicate message arrives, the
check finds the key and the consumer skips re-executing the handler.
Three design decisions in this snippet are worth calling out. First, the fingerprint is built only from fields that are present in both the original and the redelivered copy — if the broker adds or mutates fields on redelivery, a naive hash of the whole message would break deduplication. Second, the idempotency marker is stored with a TTL, so the guarantee holds only within that window; messages redelivered after the key expires are processed again, which is acceptable for most pipelines but must be a conscious decision. Third, a crash between processing and recording the key still permits a duplicate, so for truly critical handlers the marker write must be atomic with the business write — the same transactional boundary concern from the database section above.
import hashlib
import json
from typing import Optional
class MessageIdempotency:
"""Idempotency for message processing."""
def __init__(self, redis_client):
self.redis = redis_client
def process_message(self, message: dict, processor: callable) -> bool:
"""Process message with idempotency."""
# Create unique key from message
message_key = self._create_message_key(message)
# Check if already processed
if self.redis.exists(message_key):
print(f"Message already processed: {message_key}")
return True
try:
# Process message
result = processor(message)
# Mark as processed (with TTL)
self.redis.setex(message_key, 86400, json.dumps(result))
return result
except Exception as e:
print(f"Error processing message: {e}")
raise
def _create_message_key(self, message: dict) -> str:
"""Create unique key from message."""
# Use deterministic fields
key_data = json.dumps({
"type": message.get("type"),
"id": message.get("id"),
"timestamp": message.get("timestamp"),
"source": message.get("source")
}, sort_keys=True)
return f"message:processed:{hashlib.sha256(key_data).hexdigest()}"
def cleanup_old_keys(self):
"""Clean up old processed message keys."""
# Redis handles TTL automatically
pass
Using Redis as the deduplication store brings welcome properties: it is fast, survives application restarts, and is shared across all consumer instances, so duplicate messages are caught even when different workers pick them up. The TTL on the marker bounds the store’s growth and naturally expires deduplication state for old messages. The main caveat is that Redis-based checks are not transactional with the consumer’s business database, so the strongest guarantees still require moving the marker into the same transactional system that owns the side effect — the same lesson that emerged from the database-level section.
Best Practices
The mechanisms are in place, but design discipline is what separates systems that are merely correct in theory
from those that hold up in production. The final code snippet encodes the hard-won lessons of running
idempotent systems at scale as a small reference library. should_be_idempotent encodes the
method-and-resource classification discussed throughout the article — safe methods are always idempotent,
while POST operations that create financial or capacity-sensitive resources must opt in.
The key-generation helper shows another subtle point: keys should be deterministic when they describe the same
logical operation, but unique across operations. Hashing a set of natural identifiers, such as customer and
payment identifiers, produces a compact, debuggable key that is the same on every retry yet distinct from
other operations. The idempotency_key_requirements method finally documents the operational contract —
minimum length to prevent trivial collisions, a bounded maximum to keep indexes and headers small, a
restricted character set to avoid header-injection issues, and a TTL floor so clients can rely on retry
protection for a known window. Encoding these rules as code keeps every service in the organization consistent
instead of relying on tribal knowledge.
class IdempotencyBestPractices:
"""Guidelines for idempotent API design."""
@staticmethod
def should_be_idempotent(method: str, resource: str) -> bool:
"""Determine if operation should be idempotent."""
idempotent_methods = ['GET', 'PUT', 'DELETE', 'HEAD', 'OPTIONS']
# GET is always idempotent
if method in idempotent_methods:
return True
# POST operations that create resources should use idempotency
if method == 'POST':
return resource in [
'/payments', '/orders', '/subscriptions',
'/transfers', '/bookings'
]
return False
@staticmethod
def generate_idempotency_key(parts: list[str]) -> str:
"""Generate idempotency key from parts."""
combined = '-'.join(str(p) for p in parts)
return hashlib.sha256(combined.encode()).hexdigest()[:32]
@staticmethod
def idempotency_key_requirements() -> dict:
"""Requirements for idempotency keys."""
return {
"min_length": 16,
"max_length": 256,
"format": "alphanumeric with dashes/underscores",
"uniqueness": "one per user per operation",
"ttl": "24 hours minimum"
}
A common thread runs through all of the techniques in this article: correctness is achieved by making the idempotency decision explicit and consistent at every layer. Pick one source of truth for idempotency state, record responses rather than mere acknowledgements, tie the record write to the business write, and document the contract for clients so they generate stable keys. The investment pays for itself the first time a network partition, a broken client, or an aggressive retry policy would otherwise double-charge a customer or corrupt a database.
Conclusion
Idempotency is crucial for building reliable distributed systems. By implementing proper idempotency handling, you can safely handle retries without worrying about duplicate operations.
Key takeaways:
- Use idempotency keys for all state-changing POST operations
- Implement optimistic locking for updates with ETags
- Store idempotency records with appropriate TTL
- Generate descriptive idempotency keys for debugging
- Consider idempotency at all layers: API, database, and message queues
Comments