Python HMAC Authentication — Deep Dive

Production-grade HMAC implementation in Python — webhook security, API request signing, key rotation, and defense against real-world attacks.

HMAC Construction Internals

RFC 2104 defines HMAC as:

HMAC(K, m) = H((K' ⊕ opad) || H((K' ⊕ ipad) || m))

Where K' is the key processed to match the hash function’s block size (64 bytes for SHA-256), ipad is 0x36 repeated, and opad is 0x5c repeated. If the key exceeds the block size, it’s first hashed down with H(K).

This construction achieves two things: the inner hash binds the message to the key, and the outer hash prevents length extension by hashing the inner result with a different key-derived value.

import hmac
import hashlib

# These produce identical results:
# Method 1: hmac module
mac1 = hmac.new(b"secret", b"message", hashlib.sha256).hexdigest()

# Method 2: manual construction (educational, never do this in production)
import struct

key = b"secret"
block_size = 64  # SHA-256 block size

if len(key) > block_size:
    key = hashlib.sha256(key).digest()
key = key.ljust(block_size, b'\x00')

o_key_pad = bytes(k ^ 0x5c for k in key)
i_key_pad = bytes(k ^ 0x36 for k in key)

inner = hashlib.sha256(i_key_pad + b"message").digest()
mac2 = hashlib.sha256(o_key_pad + inner).hexdigest()

assert mac1 == mac2  # Identical

Production Webhook Verification

GitHub Webhooks

import hmac
import hashlib
from fastapi import Request, HTTPException

GITHUB_WEBHOOK_SECRET = b"whsec_..."  # From env/secrets manager

async def verify_github_webhook(request: Request) -> bytes:
    signature_header = request.headers.get("X-Hub-Signature-256")
    if not signature_header:
        raise HTTPException(401, "Missing signature")
    
    body = await request.body()
    
    expected = "sha256=" + hmac.new(
        GITHUB_WEBHOOK_SECRET, body, hashlib.sha256
    ).hexdigest()
    
    if not hmac.compare_digest(expected, signature_header):
        raise HTTPException(401, "Invalid signature")
    
    return body

Critical details: use request.body() for raw bytes, not request.json(). The signature covers the exact byte sequence, including whitespace and key ordering.

Stripe Webhooks

import hmac
import hashlib
import time

def verify_stripe_signature(payload: bytes, sig_header: str, 
                             secret: str, tolerance: int = 300) -> bool:
    """Verify Stripe webhook with timestamp tolerance."""
    elements = dict(
        item.split("=", 1) for item in sig_header.split(",")
    )
    timestamp = elements.get("t")
    signatures = [
        v for k, v in 
        (item.split("=", 1) for item in sig_header.split(","))
        if k == "v1"
    ]
    
    if not timestamp or not signatures:
        return False
    
    # Reject stale webhooks (replay protection)
    if abs(time.time() - int(timestamp)) > tolerance:
        return False
    
    signed_payload = f"{timestamp}.".encode() + payload
    expected = hmac.new(
        secret.encode(), signed_payload, hashlib.sha256
    ).hexdigest()
    
    return any(hmac.compare_digest(expected, sig) for sig in signatures)

Stripe includes a timestamp in the signed payload, providing built-in replay protection. The tolerance window (default 5 minutes) prevents an attacker from reusing captured webhooks after the window expires.

API Request Signing

AWS Signature Version 4, used by all AWS services, is built on HMAC-SHA256. The signing process chains multiple HMAC operations:

import hmac
import hashlib
from datetime import datetime, timezone

def aws_style_signing_key(secret: str, date: str, 
                           region: str, service: str) -> bytes:
    """Derive a signing key using chained HMAC (AWS SigV4 pattern)."""
    k_date = hmac.new(
        f"AWS4{secret}".encode(), date.encode(), hashlib.sha256
    ).digest()
    k_region = hmac.new(k_date, region.encode(), hashlib.sha256).digest()
    k_service = hmac.new(k_region, service.encode(), hashlib.sha256).digest()
    k_signing = hmac.new(
        k_service, b"aws4_request", hashlib.sha256
    ).digest()
    return k_signing

def sign_request(signing_key: bytes, string_to_sign: str) -> str:
    return hmac.new(
        signing_key, string_to_sign.encode(), hashlib.sha256
    ).hexdigest()

The chained HMAC design means even if a derived key leaks, it’s scoped to a specific date/region/service combination and can’t sign requests for other services.

Building a Request Signing Middleware

import hmac
import hashlib
import time
import json

class RequestSigner:
    """Sign and verify API requests with HMAC-SHA256."""
    
    def __init__(self, secret: bytes, max_age_seconds: int = 300):
        self.secret = secret
        self.max_age = max_age_seconds
    
    def sign(self, method: str, path: str, body: bytes = b"") -> dict:
        timestamp = str(int(time.time()))
        body_hash = hashlib.sha256(body).hexdigest()
        
        canonical = f"{method}\n{path}\n{timestamp}\n{body_hash}"
        signature = hmac.new(
            self.secret, canonical.encode(), hashlib.sha256
        ).hexdigest()
        
        return {
            "X-Timestamp": timestamp,
            "X-Signature": signature,
            "X-Body-Hash": body_hash,
        }
    
    def verify(self, method: str, path: str, body: bytes,
               headers: dict) -> bool:
        timestamp = headers.get("X-Timestamp", "")
        signature = headers.get("X-Signature", "")
        
        # Replay protection
        try:
            age = abs(time.time() - int(timestamp))
        except (ValueError, TypeError):
            return False
        if age > self.max_age:
            return False
        
        body_hash = hashlib.sha256(body).hexdigest()
        canonical = f"{method}\n{path}\n{timestamp}\n{body_hash}"
        expected = hmac.new(
            self.secret, canonical.encode(), hashlib.sha256
        ).hexdigest()
        
        return hmac.compare_digest(expected, signature)

Design decisions:

  • Canonical form includes method, path, timestamp, and body hash — an attacker can’t reuse a GET signature for a POST
  • Timestamp prevents replay attacks outside the tolerance window
  • Body hash is included in the canonical string, binding the signature to the exact payload

Key Rotation Without Downtime

import hmac
import hashlib
from dataclasses import dataclass
from typing import Optional

@dataclass
class KeyVersion:
    key_id: str
    secret: bytes
    active: bool  # Only one key signs; all valid keys verify

class HMACKeyRing:
    def __init__(self, keys: list[KeyVersion]):
        self.keys = {k.key_id: k for k in keys}
        self.signing_key = next(k for k in keys if k.active)
    
    def sign(self, message: bytes) -> tuple[str, str]:
        """Returns (key_id, signature)."""
        sig = hmac.new(
            self.signing_key.secret, message, hashlib.sha256
        ).hexdigest()
        return self.signing_key.key_id, sig
    
    def verify(self, message: bytes, key_id: str, 
               signature: str) -> bool:
        key = self.keys.get(key_id)
        if key is None:
            return False
        expected = hmac.new(
            key.secret, message, hashlib.sha256
        ).hexdigest()
        return hmac.compare_digest(expected, signature)

# Rotation: add new key as active, keep old key for verification
ring = HMACKeyRing([
    KeyVersion("v2", b"new-secret-key-here!", active=True),
    KeyVersion("v1", b"old-secret-key-here!", active=False),
])

The key ring pattern: new messages are signed with the active key, but verification accepts any key in the ring. After the old key’s maximum message age expires, remove it from the ring.

Timing Attack: Practical Demonstration

import hmac
import hashlib
import time
import statistics

def vulnerable_verify(expected: str, received: str) -> bool:
    """VULNERABLE: short-circuit comparison."""
    return expected == received

def measure_comparison(expected: str, guess: str, 
                        rounds: int = 10000) -> float:
    times = []
    for _ in range(rounds):
        start = time.perf_counter_ns()
        vulnerable_verify(expected, guess)
        elapsed = time.perf_counter_ns() - start
        times.append(elapsed)
    return statistics.median(times)

# An attacker measures timing for each character position
# Correct prefix = slightly longer comparison time
# This leaks ~1 byte per measurement campaign

In practice, this attack works on local networks with sub-millisecond latency. Over the internet, jitter adds noise but doesn’t eliminate the signal — statistical methods (Mann-Whitney U test, quartile analysis) can extract it from thousands of samples. hmac.compare_digest eliminates this vector completely.

Performance Optimization

For high-throughput systems processing thousands of webhook verifications per second:

import hmac
import hashlib

# hmac.digest() is faster than hmac.new().digest() for one-shot use
# It avoids creating an intermediate HMAC object
sig = hmac.digest(key, message, "sha256")  # Returns bytes directly

# For repeated verification with the same key, pre-compute the inner/outer keys
# by creating a template HMAC and copying it
template = hmac.new(key, digestmod=hashlib.sha256)

def fast_verify(message: bytes, expected: bytes) -> bool:
    h = template.copy()
    h.update(message)
    return hmac.compare_digest(h.digest(), expected)

The hmac.digest() function (Python 3.7+) uses an optimized C path that avoids Python object allocation overhead, making it 2–3x faster than hmac.new(key, msg, algo).digest() for small messages.

Common Pitfalls

Signing parsed data instead of raw bytes. If you parse JSON, modify it, and re-serialize, the byte sequence changes. Always sign and verify the raw wire format.

Using string comparison. Even if computed == received: in Python is vulnerable. Always hmac.compare_digest().

Hardcoding keys in source. Webhook secrets in code end up in Git history. Use environment variables, AWS Secrets Manager, HashiCorp Vault, or similar.

Ignoring replay attacks. HMAC proves authenticity but not freshness. Without a timestamp or nonce in the signed payload, an attacker can replay captured requests indefinitely.

Mixing encodings. If the sender signs UTF-8 bytes and you verify Latin-1 bytes, the signatures won’t match for any non-ASCII content. Agree on encoding explicitly.

The one thing to remember: HMAC is the bridge between “this data is intact” and “this data is genuine” — and Python’s hmac module, combined with compare_digest, gives you a production-ready implementation in a few lines.

pythonsecuritycryptography

See Also

Related Topics

ab-testing How tech companies run thousands of experiments at once to improve their products — the scientific method applied to everything from button colors to recommendation algorithms. 3 levels → activation-functions Why neural networks need these tiny mathematical functions — and how ReLU's simplicity accidentally made deep learning possible. 3 levels → ai-agents-architecture How AI systems go from answering questions to actually doing things — the design patterns that turn language models into autonomous agents that browse, code, and plan. 3 levels →