Connection Management — Deep Dive

The full connection lifecycle

A production connection goes through: creation → handshake → active use → idle → reuse → health check → retirement. Each stage has failure modes that your code must handle. This guide covers HTTP and database connection management with concrete Python implementations.

httpx connection pool internals

When you create an httpx.Client, it initializes a connection pool managed by httpcore:

import httpx

client = httpx.Client(
    limits=httpx.Limits(
        max_connections=100,       # Total connections across all hosts
        max_keepalive_connections=20,  # Idle connections to keep alive
        keepalive_expiry=30.0,     # Seconds before closing idle connections
    ),
    timeout=httpx.Timeout(
        connect=5.0,    # Time to establish connection
        read=30.0,      # Time to read response
        write=5.0,      # Time to send request
        pool=10.0,      # Time to wait for a pool connection
    ),
)

The pool timeout is often overlooked. It controls how long a request waits when all connections are in use. Without it, requests queue indefinitely when the pool is exhausted. A PoolTimeout exception is far better than a hung service.

The keepalive_expiry should be shorter than the server’s idle timeout. If your server closes idle connections after 60 seconds, set keepalive_expiry to 30 seconds. This prevents using connections the server has already closed.

requests.Session pool configuration

requests uses urllib3 connection pools under the hood. You can tune them via transport adapters:

import requests
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter


class TunedAdapter(HTTPAdapter):
    def __init__(
        self,
        pool_connections: int = 10,
        pool_maxsize: int = 20,
        max_retries: int = 0,
        pool_block: bool = True,
    ):
        super().__init__(
            pool_connections=pool_connections,
            pool_maxsize=pool_maxsize,
            max_retries=max_retries,
            pool_block=pool_block,
        )


session = requests.Session()
adapter = TunedAdapter(pool_connections=10, pool_maxsize=50, pool_block=True)
session.mount("https://", adapter)
session.mount("http://", adapter)

• pool_connections: number of connection pools (one per host)
• pool_maxsize: max connections per host
• pool_block: if True, block when pool is full instead of creating unbounded connections

Setting pool_block=True is critical for production. Without it, urllib3 creates connections beyond pool_maxsize and discards them after use — defeating connection reuse.

Async connection management with httpx

Async clients require careful lifecycle management because they’re tied to an event loop:

import httpx
from contextlib import asynccontextmanager


@asynccontextmanager
async def managed_client():
    async with httpx.AsyncClient(
        limits=httpx.Limits(
            max_connections=100,
            max_keepalive_connections=30,
            keepalive_expiry=25.0,
        ),
        timeout=httpx.Timeout(connect=5.0, read=30.0, write=5.0, pool=10.0),
    ) as client:
        yield client


# In FastAPI or Starlette
from contextlib import asynccontextmanager as acm
from fastapi import FastAPI


@acm
async def lifespan(app: FastAPI):
    app.state.http_client = httpx.AsyncClient(
        limits=httpx.Limits(max_connections=100, max_keepalive_connections=30),
        timeout=httpx.Timeout(connect=5.0, read=30.0, pool=10.0),
    )
    yield
    await app.state.http_client.aclose()


app = FastAPI(lifespan=lifespan)

The lifespan pattern ensures the client is created at startup and closed at shutdown. Using aclose() on an AsyncClient drains all pending connections gracefully.

Database connection pool with SQLAlchemy

SQLAlchemy’s connection pool is the most sophisticated in the Python ecosystem:

from sqlalchemy import create_engine, text

engine = create_engine(
    "postgresql+psycopg2://user:pass@db:5432/myapp",
    pool_size=10,           # Persistent connections
    max_overflow=20,        # Burst connections above pool_size
    pool_timeout=30,        # Wait time for a free connection
    pool_recycle=1800,      # Recreate connections after 30 minutes
    pool_pre_ping=True,     # Health check before using a connection
    echo_pool="debug",      # Log pool events (disable in production)
)

pool_pre_ping sends a lightweight query (SELECT 1) before using a connection. This catches connections that were closed by the server, killed by a firewall, or broken during a failover. The overhead is minimal (< 1ms on local connections) and prevents the dreaded OperationalError: server closed the connection unexpectedly.

pool_recycle prevents connections from growing stale. MySQL’s default wait_timeout is 8 hours, but load balancers and firewalls may close connections sooner. Setting pool_recycle to 30 minutes is a safe default.

Connection pool monitoring

You cannot manage what you cannot measure. Expose pool metrics:

import httpx
import logging

logger = logging.getLogger("connection_pool")


def log_pool_stats(client: httpx.Client) -> dict:
    pool = client._transport._pool  # type: ignore
    connections = pool.connections
    stats = {
        "total": len(connections),
        "idle": sum(
            1 for c in connections
            if hasattr(c, "is_idle") and c.is_idle()
        ),
        "active": sum(
            1 for c in connections
            if hasattr(c, "is_idle") and not c.is_idle()
        ),
    }
    logger.info("Pool stats: %s", stats)
    return stats

For SQLAlchemy:

from sqlalchemy import event


@event.listens_for(engine, "checkout")
def on_checkout(dbapi_conn, connection_record, connection_proxy):
    pool = engine.pool
    logger.info(
        "Pool checkout: size=%d, checked_in=%d, overflow=%d",
        pool.size(),
        pool.checkedin(),
        pool.overflow(),
    )

Alert when:

• Pool utilization exceeds 80% sustained
• Checkout wait time exceeds 5 seconds
• Connection creation rate spikes (indicates pool thrashing)

Graceful shutdown

Long-running services must drain connections during shutdown:

import signal
import sys
import httpx


client = httpx.Client(
    limits=httpx.Limits(max_connections=50, max_keepalive_connections=20)
)


def shutdown_handler(signum: int, frame) -> None:
    logger.info("Shutting down, closing connection pool...")
    client.close()
    sys.exit(0)


signal.signal(signal.SIGTERM, shutdown_handler)
signal.signal(signal.SIGINT, shutdown_handler)

Without graceful shutdown, Kubernetes health checks may report the pod as healthy while connections are being torn down, leading to request failures during rolling deploys.

Connection management anti-patterns

Anti-pattern 1: Client per request

# BAD: Creates a new pool for every single call
def get_user(user_id: int) -> dict:
    response = httpx.get(f"https://api.example.com/users/{user_id}")
    return response.json()

Anti-pattern 2: Unbounded pool

# BAD: No limits means uncontrolled connection growth
client = httpx.Client(limits=httpx.Limits(max_connections=None))

Anti-pattern 3: Missing pool timeout

# BAD: Requests wait forever when pool is exhausted
client = httpx.Client(
    limits=httpx.Limits(max_connections=10),
    timeout=httpx.Timeout(30.0),  # No pool timeout set
)

Anti-pattern 4: Ignoring connection errors on reuse

# BAD: Assumes pooled connections are always valid
def call_api():
    resp = session.get(url)  # Connection may have been closed server-side
    return resp.json()  # ConnectionError here is unhandled

Connection management across Python frameworks

Framework	HTTP Client	Pool Config	Lifecycle
FastAPI	httpx.AsyncClient	httpx.Limits	Lifespan event
Django	requests.Session	HTTPAdapter	Middleware or app config
Flask	requests.Session	HTTPAdapter	App context teardown_appcontext
Celery	httpx.Client	Per-worker singleton	Worker init signal

DNS caching and connection reuse

When the DNS record for a host changes (e.g., during a deployment or failover), pooled connections still point to the old IP. Solutions:

• Set pool_recycle / keepalive_expiry shorter than the DNS TTL
• Use httpx’s built-in DNS resolution (respects TTL)
• For requests, connections in the pool persist until closed — consider periodic pool cycling

This is particularly important for services behind AWS ALB or similar load balancers where IPs rotate regularly.

The one thing to remember: Production connection management combines pool sizing, health checks, timeout configuration, graceful shutdown, and monitoring — because a poorly managed connection pool causes cascading failures that are far worse than the overhead of creating new connections.