Connection Management — Core Concepts

The cost of a new connection

Every TCP connection starts with a three-way handshake (SYN → SYN-ACK → ACK). If the connection uses TLS (and it should), add another round-trip for the TLS handshake. On a typical internet connection, that’s 100-300ms before any data flows. For internal services on the same network, it’s still 1-5ms — time that adds up fast when you’re making hundreds of calls per second.

Connection pooling explained

A connection pool maintains a set of pre-established connections that your code borrows and returns. The flow is:

1. Code requests a connection to api.example.com
2. Pool checks if an idle connection exists
3. If yes, return it immediately (no handshake)
4. If no, create a new one (up to the pool limit)
5. When the code finishes, the connection returns to the pool

Both requests.Session and httpx.Client manage connection pools internally. The key difference from bare requests.get() is that bare calls create a new session (and pool) per call — destroying the pool after each use.

HTTP keep-alive

HTTP/1.1 introduced persistent connections by default. The Connection: keep-alive header tells the server “don’t close this connection after responding.” Both sides can send multiple request-response pairs over the same TCP connection.

Without keep-alive, every HTTP request needs a fresh TCP connection. With it, the second request to the same host reuses the existing connection — skipping the entire handshake.

Pool sizing

The right pool size depends on your workload:

• Too small: Requests queue up waiting for a free connection, adding latency
• Too large: You hold open connections that consume server resources (and hit OS file descriptor limits)

A starting point: set max_connections to your expected concurrent request count plus a 20% buffer. For most Python services, 20-50 connections per host is reasonable. High-throughput services might need 100+.

Monitor pool utilization: if connections are always at the limit, increase it. If most connections sit idle, decrease it.

Connection lifecycle

Connections don’t last forever. They degrade due to:

• Server timeouts — servers close idle connections (typically after 60-120 seconds)
• Network changes — roaming between networks, VPN reconnects
• Load balancer rotation — the server behind the connection may have been removed from the pool
• TCP RST packets — firewalls or intermediaries can kill idle connections silently

Good connection management includes health checks: validate a connection before using it, and replace it if the check fails. Most HTTP libraries handle this transparently, but database connection pools (like SQLAlchemy’s pool) offer explicit pool_pre_ping settings.

Context managers prevent leaks

The most common connection management bug is forgetting to close. Every httpx.Client(), requests.Session(), database engine, or socket should be used with a context manager:

with httpx.Client() as client:
    response = client.get("https://api.example.com/data")

Without with, a crashed or early-returning function leaves connections open until Python’s garbage collector eventually cleans them up — which might be never in long-running processes.

Database connection pools

The same principles apply to database connections, where the handshake cost is even higher (authentication, session setup, character encoding negotiation). SQLAlchemy, Django ORM, and asyncpg all use connection pools. Key parameters:

• pool_size — number of persistent connections
• max_overflow — extra connections allowed during bursts
• pool_timeout — how long to wait for a free connection before raising an error
• pool_recycle — maximum lifetime of a connection (prevents stale connections)

Common misconception

Many developers think creating an httpx.Client() or requests.Session() at the module level is wrong because “it holds resources.” In reality, a module-level client is the correct pattern for long-running services. Creating a new client per request defeats the entire purpose of connection pooling. The client should live as long as the process — create it once during startup and close it during shutdown.

The one thing to remember: Connection management isn’t just about performance — it’s about preventing resource exhaustion, handling network instability, and ensuring your Python service stays healthy under load.